Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
f66cd4549fb5...
f66cd4549fb55830443ebd2f9df4713d8f7281cd
Sergei Golubchik
MDEV-40340 mariadb-import --lock-tables crashes

don't change `argv` pointer, it's needed later for --lock-tables
dee8add860cd...
dee8add860cdca409ab3313209f182b9acf7073b
Sergei Golubchik
memory leak in mariadb-import --lock-tables
d8e582437eb6...
d8e582437eb61e9db15cd4ac177cf87158dbb2c1
Monty
MDEV-25292 Atomic CREATE OR REPLACE TABLE

Atomic CREATE OR REPLACE allows to keep an old table intact if the
command fails or during the crash. That is done by renaming the
original table to temporary name, as a backup and restoring it if the
CREATE fails. When the command is complete and logged the backup
table is deleted.

Atomic replace algorithm

  Two DDL chains are used for CREATE OR REPLACE:
  ddl_log_state_create (C) and ddl_log_state_rm (D).

  1. (C) Log rename of ORIG to TMP table (Rename TMP to original).
  2. Rename orignal to TMP.
  3. (C) Log CREATE_TABLE_ACTION of ORIG (drops ORIG);
  4. Do everything with ORIG (like insert data)
  5. (D) Log drop of TMP
  6. Write query to binlog (this marks (C) to be closed in
    case of failure)
  7. Execute drop of TMP through (D)
  8. Close (C) and (D)

  If there is a failure before 6) we revert the changes in (C)
  Chain (D) is only executed if 6) succeded (C is closed on
  crash recovery).

Foreign key errors will be found at the 1) stage.

Additional notes

  - CREATE TABLE without REPLACE and temporary tables is not affected
    by this commit.
    set @@drop_before_create_or_replace=1 can be used to
    get old behaviour where existing tables are dropped
    in CREATE OR REPLACE.

  - CREATE TABLE is reverted if binlogging the query fails.

  - Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
    this commit. Conflicting tables marked with this flag will be
    deleted with CREATE OR REPLACE.

  - Replication execution is not affected by this commit.
    - Replication will first drop the conflicting table and then
      creating the new one.

  - CREATE TABLE .. SELECT XID usage is fixed and now there is no need
    to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
    do_postlock()). XID is now correctly updated so it disables
    DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
    final stage when the table is ready. So if we have XID in the
    binary log we don't need to drop the table.

  - Three variations of CREATE OR REPLACE handled:

    1. CREATE OR REPLACE TABLE t1 (..);
    2. CREATE OR REPLACE TABLE t1 LIKE t2;
    3. CREATE OR REPLACE TABLE t1 SELECT ..;

  - Test case uses 6 combinations for engines (aria, aria_notrans,
    myisam, ib, lock_tables, expensive_rename) and 2 combinations for
    binlog types (row, stmt). Combinations help to check differences
    between the results. Error failures are tested for the above three
    variations.

  - expensive_rename tests CREATE OR REPLACE without atomic
    replace. The effect should be the same as with the old behaviour
    before this commit.

  - Triggers mechanism is unaffected by this change. This is tested in
    create_replace.test.

  - LOCK TABLES is affected. Lock restoration must be done after new
    table is created or TMP is renamed back to ORIG

  - Moved ddl_log_complete() from send_eof() to finalize_ddl(). This
    checkpoint was not executed before for normal CREATE TABLE but is
    executed now.

  - CREATE TABLE will now rollback also if writing to the binary
    logging failed. See rpl_gtid_strict.test

backup ddl log changes

- In case of a successfull CREATE OR REPLACE we only log
  the CREATE event, not the DROP TABLE event of the old table.

ddl_log.cc changes

  ddl_log_execute_action() now properly return error conditions.
  ddl_log_disable_entry() added to allow one to disable one entry.
  The entry on disk is still reserved until ddl_log_complete() is
  executed.

On XID usage

  Like with all other atomic DDL operations XID is used to avoid
  inconsistency between master and slave in the case of a crash after
  binary log is written and before ddl_log_state_create is closed. On
  recovery XIDs are taken from binary log and corresponding DDL log
  events get disabled.  That is done by
  ddl_log_close_binlogged_events().

On linking two chains together

  Chains are executed in the ascending order of entry_pos of execute
  entries. But entry_pos assignment order is undefined: it may assign
  bigger number for the first chain and then smaller number for the
  second chain. So the execution order in that case will be reverse:
  second chain will be executed first.

  To avoid that we link one chain to another. While the base chain
  (ddl_log_state_create) is active the secondary chain
  (ddl_log_state_rm) is not executed. That is: only one chain can be
  executed in two linked chains.

  The interface ddl_log_link_chains() was defined in "MDEV-22166
  ddl_log_write_execute_entry() extension".

Atomic info parameters in HA_CREATE_INFO

  Many functions in CREATE TABLE pass the same parameters. These
  parameters are part of table creation info and should be in
  HA_CREATE_INFO (or whatever). Passing parameters via single
  structure is much easier for adding new data and
  refactoring.

InnoDB changes
  Added ha_innobase::can_be_renamed_to_backup() to check if
  a table with foreign keys can be renamed.

Aria changes:
- Fixed issue in Aria engine with CREATE + locked tables
  that data was not properly commited in some cases in
  case of crashes.

Other changes:
- Removed some auto variables in log.cc for better code readability.
- Fixed old bug that CREATE ... SELECT would not be able to auto repair
  a table that is part of the SELECT.
- Marked MyISAM that it does not support ROLLBACK (not required but
  done for better consistency with other engines).

Known issues:
- InnoDB tables with foreign key definitions are not fully supported
  with atomic create and replace:
  - ha_innobase::can_be_renamed_to_backup() can detect some cases
    where InnoDB does not support renaming table with foreign key
    constraints.  In this case MariaDB will drop the old table before
    creating the new one.
    The detected cases are:
    - The new and old table is using the same foreign key constraint
      name.
    - The old table has self referencing constraints.
  - If the old and new table uses the same name for a constraint the
    create of the new table will fail. The orignal table will be
    restored in this case.
  - The above issues will be fixed in a future commit.
- CREATE OR REPLACE TEMPORARY table is not full atomic. Any conflicting
  table will always be dropped before creating a new one. (Old behaviour).

Bug fixes related to this MDEV:

MDEV-36435 Assertion failure in finalize_locked_tables()
MDEV-36439 Assertion `thd_arg->lex->sql_command != SQLCOM_CREATE_SEQUENCE...
MDEV-36498 Failed CoR in non-atomic mode no longer generates DROP in RBR...
MDEV-36508 Temporary files #sql-create-....frm occasionally stay after
          crash recovery
MDEV-38479 Crash in CREATE OR REPLACE SEQUENCE when new sequence cannot
          be created
MDEV-36497 Assertion failure after atomic CoR with Aria under lock in
          transactional context
MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
          table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge
MDEV-39367 MSAN/valgrind errors in temp_file_size_cb_func,
          main.tmp_space_usage fails
MDEV-39446 Atomic CREATE OR REPLACE fails if a table cannot be decrypted

InnoDB related changes:
- ha_innodb::rename_table() does not handle foreign key constraint
  when renaming an normal table to internal tempory tables. This
  causes problems for CREATE OR REPLACE as the old constraints causes
  failure when creating a new table with the same constraints.
  This is fixed inside InnoDB by not threating tempfiles (#sql-create-..),
  created as part of CREATE OR REPLACE, as temporary files.
- In ha_innobase::delete_table(), ignore checking of constraints when
  dropping a #sql-create temporary table.
- In tablename_to_filename() and filename_to_tablename(), don't do
  filename conversion for internal temporary tables (#sql-...)

Other things:
- maria_create_trn_for_mysql() does not register a new transaction
  handler for commits. This was needed to ensure create or replace
  will not end with an active transaction.
- We do not get anymore warnings about "Engine not supporting atomic
  create" when doing a legal CREATE OR REPLACE on a table with
  foreign key constraints.
- Updated VIDEX engine flags to disable CREATE SEQUENCE.

Reverted commits:
MDEV-36685 "CREATE-SELECT may lose in binlog side-effects of
stored-routine" as it did not take into account that it safe to clear
binlogs if the created table is non transactional and there are no
other non transactional tables used.
- This was done because it caused extra logging when it is not needed
  (not using any non transactional tables) and it also did not solve
  side effects when using statement based loggging.

Other things:
- EITS data is preserved if create or replace fails if
  drop_before_create_or_replace=OFF. If ON, then create or replace
  will drop EITS before the drop of the original table (as before).
- Using CREATE OR REPLACE on a encrypted table that the user cannot
  decrypt will fail instead of replacing the encrypted table.
  The encrypted table will unchanged.
c45ab9796651...
c45ab979665137f530848ff429f3820c19421471
Rex Johnston
MDEV-39492 Parallel Query: workers evaluate with the session's variables

A worker runs in a background THD, which starts from the global variable values,
and init_parallel_workers() copied only the identity of the session: its
security context, database and command. So the workers evaluated the session's
own expressions with somebody else's settings, and the query quietly meant
something different there. With the session in one time zone and the server in
another,

  SET TIME_ZONE = "+03:00";
  SELECT HOUR(ts) FROM t1;

answered 3 in this thread and 12 in a worker, and the same difference inside a
WHERE dropped every row a condition on a TIMESTAMP column should have kept, which
is how function_defaults_innodb failed with parallel_worker_threads forced on.

Hand each worker the variables an expression reads while it is evaluated:
time_zone, sql_mode, default_week_format and old_behavior. One at a time rather
than as a whole struct, because system_variables owns per-THD allocations, the
dynamic variables and the session tracker among them, and it also carries
option_bits, which would tell a worker it is inside the session's
multi-statement transaction and change how it commits.

Which variables those are was settled by testing rather than by listing what
looked relevant. Variables read while an expression is *built* need no handover,
because the copies are built and fixed on the user's thread: DAYNAME(),
MONTHNAME() and DATE_FORMAT('%W') keep the lc_time_names they were fixed with,
and a division keeps the scale div_precincrement gave it, so neither variable is
copied. max_allowed_packet cannot be set per session, so a worker already has the
session's value. That leaves the four above, three of them because a test showed
each one changing an answer, and old_behavior because the date and time
conversions read it as they run, the same way sql_mode is read.

parallel_query_session_vars checks the five expressions in both places, projected
and in a WHERE. Without the handover the projection answers 12, 2 and 9 where it
should answer 3, 10 and 10, and the WHERE returns no rows where it should return
three.

This commit was prepared with Claude Code: it reduced the failure to HOUR() over
a TIMESTAMP, established which variables are read at evaluation time and which at
fix time, and wrote the test.
b9d6caa133f6...
b9d6caa133f6904d46a25edd9b01e5b8d29c0d47
Rex Johnston
MDEV-39492 Parallel Query: give ANALYZE the numbers the workers produced

ANALYZE reads a JOIN_TAB's counters from the tracker the optimizer left on the
Explain object, and the engine's counters from the handler it recorded there.
Both belong to the manager, and the manager never runs the driving table's read
loop, so a parallel query reported the table as untouched: r_loops 0, no r_rows,
no r_filtered, no r_engine_stats. For a feature whose whole purpose is to make a
scan faster, the tool for seeing where a scan spends its time said nothing about
it.

Each worker now counts what it did to each of its tables in the same terms
sub_select() and evaluate_join_record() use -- rows read, rows that passed the
table's condition, and one scan per probe of an inner table -- and copies the
engine's counters out of the tables while they are still open. The manager adds
all of it to the trackers and handlers ANALYZE reads, in quiesce_workers() after
every worker has been joined, so this thread is the only one touching either
side and no locking is needed. ha_handler_stats::add() already existed for the
partitioning case and does the engine half.

The driving table reports one scan, not one per worker: the chunks are one scan
of the table between them, which is what the serial plan reports and what keeps
the rows-per-scan figure comparable between the two.

  ANALYZE SELECT a FROM t1 WHERE a % 7 = 0;

now answers r_rows 5000.00 and r_filtered 14.28 whether it runs serially or in
the workers, differing only in the access type.

Not included: r_table_time_ms and r_other_time_ms, which come from the elapsed
time trackers rather than from counters, and which the workers measure in
parallel. Summing them would report more time against the table than the query
itself took, and reporting one worker's share would understate the work. That
needs a decision about what ANALYZE should mean for a parallel scan, so it is
left out rather than guessed at, and the two fields stay absent.

parallel_query_worker_side compares the tabular ANALYZE for the same query run
both ways, with the row estimate masked because it is an InnoDB approximation.
Without the handover the parallel run reports r_rows and r_filtered as NULL.

This commit was prepared with Claude Code.
ac8f88618262...
ac8f8861826205d9fa8ad7d953a5acc862fae4c7
Rex Johnston
MDEV-39492 Parallel Query: start no more workers than the engine has chunks

parallel_worker_threads is a request, not a division of labour. InnoDB
divides the table at B-tree boundaries: Parallel_reader partitions at the root
page, so the number of chunks is that page's fanout, and create_contexts()
declines to subdivide further while the tree is shallower than
SPLIT_THRESHOLD. A 44MB table of a million rows yields four chunks, and it
yields four whether one worker was asked for or fifty.

Workers past the last chunk were created anyway. Such a worker is handed
HA_ERR_END_OF_FILE the first time it asks for work and exits without reading a
row, having already cost a THD, a table instance opened from the share for
every table in the join, the cloned conditions and select list, and a row
buffer. On that four-chunk table, asking for twelve workers started twelve and
found work for four. The query took the same time as with four, because the
other eight had nothing to contribute.

Ask the engine how many chunks it made and start no more workers than that.
handler::pscan_chunk_count() answers 0 for an engine that cannot say, and
InnoDB answers 0 as well while any chunk is still flagged for splitting: such
a chunk is replaced at run time by a variable number of finer ones, so the
count is not an upper bound yet and must not be used as one. That is what
keeps the large-table case, where the tail chunks do get split, from being
held down to the pre-split count.

Parallel_workers_started counts the workers really started, summed over the
queries Parallel_queries_executed counts, which is what makes the difference
visible from SQL. parallel_query_worker_count reads the two as a ratio,
because both are cumulative and --ps-protocol executes a statement more than
once, so workers-per-query is the figure that holds across protocols. Without
the fix a table of a single leaf page reports eight and sixteen workers where
it now reports one.

This makes nothing faster. It stops paying for threads that cannot be given
anything to read, which matters under concurrency, where they compete for the
same cores as everyone else.

This commit was prepared with Claude Code: it measured the chunk counts and
their row distribution by instrumenting create_contexts() and
pscan_get_next_row(), confirmed the failing forced-worker tests are unchanged
from HEAD, and wrote the test.
a9309230b3cc...
a9309230b3cc176ba102f210163cff345b939ab2
Sergei Golubchik
MDEV-40411 fix the comparison

don't convert ulonglong max_mem_used to uint32 before the comparison
9d4ffe23b258...
9d4ffe23b25808f7858b2dc48106fbd60cf0e513
Sergei Golubchik
MDEV-40312 SHOW CREATE SERVER incorrect quoting

quote protocol name and option names
9d1466c8be26...
9d1466c8be263256c45e02d9b484b25438971fd8
Sergei Golubchik
MDEV-40337 disable the test in valgrind builds

valgrind doesn't like it when a process writes to r/o memory
and complains "Bad permissions for mapped region". It's not memcheck,
but valgrind core error, cannot be suppressed.
822ff6a8534f...
822ff6a8534f11d7395dc604bc21223f77903f1e
Sergei Golubchik
MDEV-40312 SHOW CREATE SERVER incorrect quoting

quote protocol name and option names
719b19a88177...
719b19a8817755b5152a8258fa7d127f123f8c32
Sergei Golubchik
MDEV-40312 SHOW CREATE SERVER incorrect quoting

quote protocol name and option names
6b939761b9b8...
6b939761b9b85846178465f5819e1dd1bd384651
bsrikanth-mariadb
MDEV-40383:innodb_gis.point_basic fails on replay

There are 2 problems: -
1. The REPLACE statement that is recorded doesn't store the
  value of geometry type field correctly.
2. The table definition that got recorded has fields with non-null constraint,
  and no default value is specified.
  Also, the "REPLACE INTO" statement that gets stored in the context,
  doesn't have any value specified for these non-null fields.

Solution is to: -
1. When using REPLACE INTO statement, store all the non-numeric values in HEX,
  whenever conversion from field's charset to output's charset is lossy.
2. Instead of storing only the column values that were projected in the
  query, store all the non-virtual column values into the recorded
  REPLACE INTO statement.

Implementation details: -
1. Introduce a new method is_charset_conversion_lossless() in filesort.cc,
  to check if the output charset to which field's data is being written to,
  results in a lossless conversion. If so, non-numeric values being witten
  using REPLACE INTO statement are stored in string representation,
  else they are converted to HEX.
2. From join_read_const(), and join_read_system() methods in sql_select.cc,
  re-read the const row for all the non-virtual fields in the table.
  After the row is re-read and recorded, restore the table->read_set,
  table->status, and the const row, to the value that was before.
657859fbf4d8...
657859fbf4d8488489960e54b9b64475cec7e358
bsrikanth-mariadb
MDEV-40388: sequence.simple fails on replay

The problem is that, when recording is enabled for the query such as,
explain select * from seq_1_to_10;
it recorded the table context having a DDL definition as: -

CREATE TABLE `seq_1_to_10` (
    ->  `seq` bigint(20) unsigned NOT NULL,
    ->  PRIMARY KEY (`seq`)
    -> ) ENGINE=SEQUENCE DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci;

Now, when that context is replayed, the DDL statement is executed.
But, we cannot create such a table, and instead it errors out saying
ERROR 1050 (42S01): Table 'seq_1_to_10' already exists.

Solution is to use: -
  CREATE TABLE IF NOT EXISTS seq_1_to_10 ...;

=====

Also, there is a different way to use sequences as: -
  Create sequence s1;
  Explain select * from s1;

Here, we should be recording the DDL statement, but no need to store the
stats for it. However, we didn't record the DDL statement earlier.
Moreover, sequence's next value should be the same in the replay environment.

Solution here is to record the DDL for such a sequence as
  CREATE TABLE IF NOT EXISTS s1 ...;
and also set its start value as the recorded environment's previous value using
  SELECT SETVAL(s1, prev_value);
6552e450613f...
6552e450613fc4b8bc67974cf7ac6b4c88eeb157
Sergei Golubchik
MDEV-40340 mariadb-import --lock-tables crashes

don't change `argv` pointer, it's needed later for --lock-tables
5d72183d1813...
5d72183d1813cc3535061d142f83b7d38b0bf34e
Rex Johnston
MDEV-39492 Parallel Query: cost by the leaf page count, not the root fanout

pscan_chunk_count_estimate() answered the root page's fanout, because that was
all a scan could be divided into. Ctx::split() now divides a coarse range
further, so the ceiling is no longer where the division starts but where it can
no longer go: a chunk cannot be smaller than a leaf page, there being nothing
under a leaf page to divide by. The estimate is therefore the clustered index's
leaf page count, which is also the right answer for a tree too shallow to be
split, where the ranges are the root's records and the root's records are the
leaf pages.

The old answer under-stated the ceiling on exactly the tables the split was
added for. A table of six thousand rows whose root holds two records was costed
as though two workers were the most it could occupy, so the optimizer stopped
believing in a third; the table has ninety-two leaf pages and the split reaches
them. Under-stating is the safe direction to be wrong in, which is why it was
left this way in the commit that added the estimate, but it is no longer
accurate, and it would have kept the optimizer from choosing plans the engine
can now execute several times faster.

The statistics are still what they are. InnoDB reports a thousand-row table as
one leaf page where its data length says four, so a table that small gets no
discount at all and is costed as the serial scan it very nearly is. That is the
same conservatism as before and it costs nothing real, the setup term dominating
anything that small either way.

The per-row factor stays at 1.16. It was measured before chunks were split and
re-measuring now gives a figure below 1, but only in the better of two regimes
the same query alternates between: at six workers the scan-only case runs at
either 19 or 31 milliseconds with nothing in between, and which one it picks
varies between runs of a few queries. Setting a constant from the good mode
would be claiming a per-row saving that is not reliably there, so the comment
now says which end of the range the number is rather than presenting it as a
measurement of the current code. The bimodality is worth its own investigation:
the fast mode implies a per-worker throughput above the serial reader's, which
would be the InnoDB prefetcher engaging, and it is a factor of 1.6.

parallel_query_worker_count now asserts on t4, whose root holds two records over
ninety-two leaf pages, that eight workers are costed below six. Six is what the
root fanout could pay for once the setup term is included, so improving past it
is the ceiling having risen. Reverting the estimate to the root fanout flips
that assertion. The t3 case, which used to carry the proof that a discount is
applied at all, now records the opposite and t4 carries the discount proof, so
neither assertion can pass vacuously.

The forced-worker sweep of the main suite is unchanged at 20 failures.

This commit was prepared with Claude Code: it derived the leaf page ceiling,
read the actual statistics out of mysql.innodb_index_stats when two of its own
assertions came out wrong, found that the setup term legitimately makes sixteen
workers dearer than four on a table this size, and settled on the pair of
assertions above after confirming by mutation that they detect the change.
5bbcd5ef8a2c...
5bbcd5ef8a2c66f8fcc2b6740869e4418aac065e
Sergei Golubchik
mysqldump: remove dead and broken code

since 2006 (3840774309bc) mysqldump tried to be smart when dumping
events - it tried to automatically detected a delimiter per event
that was not present in the event body, using ";;" by default.

This never worked, was broken since the first commit. It either used
the default ";;" or failed after trying the same ";;" delimiter
2147483646 times.

All other objects (routines, triggers, etc) used a hard-coded ";;"
for 20 years, which apparently worked fine. Let's remove the old
broken logic and dump events like all other objects,
5af9369c6205...
5af9369c6205523902b05a35e8cb7aa49f306f4c
Rex Johnston
MDEV-39492 Parallel Query: filter by the condition from before the pushdown

When the optimizer pushes part of a condition into the engine,
push_index_cond() leaves tab->select_cond holding only the remainder and keeps
the original in tab->pre_idx_push_select_cond. The pushed half lives on from
there in handler::pushed_idx_cond, which belongs to the handler it was pushed
into -- the manager's. A worker reads through its own handler, opened by
open_table_from_share() with nothing pushed into it, and it cloned select_cond,
so the pushed half was enforced in neither place.

  SELECT pk, a, b FROM p1,p2,p3 WHERE b >= d AND pk < c AND b = '0';

answered one row serially and six with workers, `pk < c` having been applied
nowhere, and the unfiltered rows then multiplied against the third table.
Setting index_condition_pushdown=off made the parallel answer correct, which is
what pinned it on the pushdown rather than on the plan the cost model chose.

Clone the pre-pushdown condition where there is one. One accessor,
pwt_table_cond(), is used by the gate and by both clone sites, so the item the
gate approves is always the item a worker evaluates -- the two drifting apart is
what this bug was.

This gives up what the pushdown was for: the engine no longer rejects an index
entry before the row is read, so a worker does more clustered-index work per
match than the serial plan. The alternative was to refuse these plans at the
gate, which would have cost the parallel scan altogether on a common plan shape.
Correct and parallel beats correct and serial here, and pushing a clone onto the
worker's own handler would recover the difference -- that wants the worker to
hold a real JOIN_TAB to hang the key number off, so it belongs with the cloned
JOIN, not before it.

parallel_query_join gains the query above, answered serially and again with
workers, asserting that it still ran in the workers rather than falling back and
that p1 still carries a pushed index condition, so the case cannot quietly stop
being covered. Without the fix the parallel answer is six rows.

range_innodb, which is where this was found with parallel_worker_threads forced
on, now differs only in EXPLAIN output.

This commit was prepared with Claude Code: it traced the predicate to
push_index_cond() moving it out of select_cond, and wrote the test.
59652e8466a3...
59652e8466a31463332f65219c9c1de4660cb583
Sergei Golubchik
mysqldump: remove dead and broken code

since 2006 (3840774309bc) mysqldump tried to be smart when dumping
events - it tried to automatically detected a delimiter per event
that was not present in the event body, using ";;" by default.

This never worked, was broken since the first commit. It either used
the default ";;" or failed after trying the same ";;" delimiter
2147483646 times.

All other objects (routines, triggers, etc) used a hard-coded ";;"
for 20 years, which apparently worked fine. Let's remove the old
broken logic and dump events like all other objects,
4391617f0216...
4391617f0216264125bed467948aaf050e3f4586
Rex Johnston
MDEV-39492 Parallel Query: cost a parallel scan by what it can actually do

scale_cost_for_parallel_scan() divided the row and copy cost by
parallel_worker_threads, with nothing else in the arithmetic. Three things were
missing and all three pointed the same way, so the optimizer was most
optimistic exactly where parallelism helps least.

The divisor was the request rather than what the table can be divided into. The
engine partitions at the root page, so the chunk count is that page's fanout,
and a worker beyond the last chunk gets nothing to read. With
parallel_worker_threads=50 on a table of a million rows in 44MB, the optimizer
believed the scan was fifty times cheaper. It is four chunks, so a little over
twice as cheap. An error of that size is enough to prefer a parallel full scan
over a perfectly good index, which is what the forced-worker runs of
greedy_optimizer, partition_explicit_prune, type_temporal_innodb, vector and
xtradb_mrr were showing.

Reading a row through a worker also costs more than reading it serially, some
1.16 times for a scan whose rows are cheap to evaluate, because every row is
copied into a batch buffer, handed over under a mutex and read again by the
manager. And each worker has to be built: a THD, a table instance per table in
the join opened from the share, the cloned items and a row buffer, some 22
microseconds. Both are measured on a release build.

So: clamp the divisor to handler::pscan_chunk_count_estimate(), multiply by the
per-row factor, and add the setup cost. The estimate comes from the clustered
index statistics rather than from a page read, since it is wanted at optimize
time: for a tree of three levels the root's fanout is every non-leaf page but
the root, and for a tree of two levels the root's records are the leaf pages.
It needs the statistics to have been gathered, and answers 1 without them, so
an un-analyzed table is costed as the serial scan it may well end up being.
That is the safe direction to be wrong in, because it never claims parallelism
the table cannot supply.

init_parallel_workers() now declines a table the engine cannot divide at all.
One chunk means one worker reading the whole table by itself, which is the
serial scan plus a copy, a handover and a re-read for every row of it. There is
nothing to parallelise and the serial path does it for less.

parallel_query_join's index-condition case needed two changes as a consequence,
and they are worth spelling out because the case had been passing for the wrong
reason. Its p2 held two rows in a single leaf page, so the engine cannot divide
it and the query now runs serially; p2 grows to more than one page. Its plan
also depended on the old cost model: a two-row table was only ever the driving
table because dividing its scan cost by four made it look cheaper than reading
p1 by its index. With the cost fixed the optimizer puts p1 first, where pk < c
cannot be pushed into an index at all, so the pushed-condition situation the
test exists to cover disappeared from the plan. STRAIGHT_JOIN now pins the
order the case needs. Reverting pwt_table_cond() under the new test answers
three thousand rows where one is correct, so the coverage is intact and rather
sharper than the six rows it used to produce.

The forced-worker sweep of the main suite goes from 25 failures to 20, the five
above, and adds none.

This commit was prepared with Claude Code: it measured the per-row factor and
the setup cost, derived the chunk-count estimate and checked it against
instrumented chunk counts on two tables, found that the index-condition case
had been relying on the mis-costing, and confirmed by mutation that both halves
of this change are what the tests detect.
37e67dd4dc75...
37e67dd4dc7569fe9a29236294de3c69c0b65045
Sergei Golubchik
MDEV-40340 mariadb-import --lock-tables crashes

don't change `argv` pointer, it's needed later for --lock-tables
2a4cfdde2bcc...
2a4cfdde2bcc1eddf87c36c19a25381f278c3956
Sergei Golubchik
mysqldump: remove dead and broken code

since 2006 (3840774309bc) mysqldump tried to be smart when dumping
events - it tried to automatically detected a delimiter per event
that was not present in the event body, using ";;" by default.

This never worked, was broken since the first commit. It either used
the default ";;" or failed after trying the same ";;" delimiter
2147483646 times.

All other objects (routines, triggers, etc) used a hard-coded ";;"
for 20 years, which apparently worked fine. Let's remove the old
broken logic and dump events like all other objects,
2894e90b6545...
2894e90b6545d3c7fa13ff00fb50c9e8631003e7
Sergei Golubchik
MDEV-40165 post-fix

never use item->null_value before this item is evaluated

followup for 14c16e02b26
1bf11ed89121...
1bf11ed89121c13178ef5dcf07ccb9b01f41a457
Sergei Golubchik
MDEV-40340 mariadb-import --lock-tables crashes

don't change `argv` pointer, it's needed later for --lock-tables
1a572838e246...
1a572838e246d8f156b286b8220428b2f0824c8c
Sergei Golubchik
MDEV-40554 KILL checks user (not priv_user) and doesn't verify hostname

The "is it my own thread" check compared the login user name
(Security_context::user) and ignored the host, so u1@localhost could see
and kill threads of u1@'127.0.0.1'.

Compare the authenticated account instead:

* all comparisons are done in sctx->is_priv_user() now
* change user_matches() to priv_user_matches(), which uses is_priv_user()
* use it in KILL, KILL USER, SHOW PROCESSLIST, I_S.PROCESSLIST,
  COM_PROCESS_INFO and SHOW EXPLAIN/ANALYZE FOR.
* all the remaining places use is_priv_user() directly instead
  of doing strcmp: SHOW GRANTS, SHOW CREATE PROCEDURE, I_S.VIEWS,
  the DEFINER clause, optimizer trace and change_security_context().

Assisted-by: Claude:claude-5-opus
19bcbb103966...
19bcbb1039662b9c64765f74c1de57d40c74569b
Rex Johnston
MDEV-39492 Parallel Query: bound the split, and use it where it is needed

A chunk came from a whole B-tree level, so a scan could be divided two ways and
no other. The root's fanout gave the coarse ranges, and Ctx::split() re-divided
one of those at the level above the leaves, producing a range per leaf page. On
a table of a million rows in 44MB that is a choice between 4 chunks and 2245,
where what the query wants is a few dozen. create_contexts() therefore declined
to split at all unless the tree was at least SPLIT_THRESHOLD deep, which for a
16K page means about 16GB, so in practice a table was left at its root fanout
however lopsided that was. Four chunks measured 36.0, 36.0, 18.0 and 10.0 per
cent of the rows: the largest alone held the scan open for a third of its
serial duration, and no number of workers changed that.

A range ends where the next one begins, so leaving out a start point folds that
sub-tree into the range before it and loses no rows. create_ranges() now takes
a bound on how many ranges a page may yield and strides its records to meet it,
skipping the descent into the sub-trees it folds -- not descending is the
saving. Ctx::split() asks for as many pieces as there are threads. The bound is
what makes splitting cheap enough to rely on: each piece costs a page traversal,
a context and a deep-copied tuple, built under the index S-latch, and at leaf
granularity that outweighed the balance it bought on any scan short enough to
notice.

So the depth test goes. What replaces it is structural rather than a heuristic:
a range is split unless there is nothing under the root to divide it by, which
is m_depth below 2, meaning the ranges already end at leaf pages. Those are the
trees of one or two levels, and they have at most about a thousand leaf pages,
which are their chunks already -- more than enough, and leaving them alone also
keeps the count final so init_parallel_workers() can still size its pool by it.

Measured on a release build, six cores pinned, medians of 27 samples with the
worker counts interleaved so no configuration gets a turbo window. A scan of a
million rows returning one, which is the case that suffers most from a coarse
chunk because it has no other work to hide behind, goes from 2.10 to 5.49 times
serial. The same scan joined to two dimension tables goes from 1.76 to 3.16.
Splitting the same table without the bound reached 1.93 and 2.61 and made the
scan-only case erratic, its inter-quartile range 27 per cent against 1 to 2 for
the others. A join over a thousand-row driving table is untouched, its tree
being two levels deep. COUNT, SUM and an order-independent CRC over both
million-row tables agree with the serial answer at 1, 3, 5, 12 and 50 workers,
so the folded ranges cover every row exactly once.

Parallel_scan_chunks reports what the table was divided into, which is what
makes the bound testable rather than only measurable: the new case in
parallel_query_worker_count is a table whose root holds two records and whose
tree has a level between root and leaves. It starts all eight workers asked for
where before it started two, and it divides into ten chunks. Removing the bound
divides the same table into 56, one per leaf page, and restoring the depth test
starts two workers again.

Not addressed: a split can still produce fewer pieces than there are threads,
if the page it strides has fewer records than that. The pool has already been
sized by then and the count is not knowable earlier, so a worker can still come
away with nothing. It costs a THD rather than a wrong answer.

This commit was prepared with Claude Code: it established that chunk boundaries
are B-tree level boundaries and measured the resulting chunk sizes by counting
rows per chunk in pscan_get_next_row(), wrote the striding, took the
measurements above, and confirmed by mutation that the test detects both the
loss of the bound and the loss of the split.
160e247704ba...
160e247704bab3ccb7de87aa8913cc1dae6dbc20
Sergei Golubchik
memory leak in mariadb-import --lock-tables
0ccc90818fee...
0ccc90818fee0ecb63884babbca9a26aeb7bf932
Sergei Golubchik
CREATE SERVER: fix option parsing to support backticks properly

CREATE SERVER used to:
* support arbitrary options in backticks and not in backticks
* hard-coded historical options worked *only* without backticks
* PORT range was different when given as a number or a string

all that is fixed, unused keywords are removed
09bc25339b6d...
09bc25339b6dc70cb2a457053c94f474b87e8f4c
Sergei Golubchik
CREATE SERVER: fix option parsing to support backticks properly

CREATE SERVER used to:
* support arbitrary options in backticks and not in backticks
* hard-coded historical options worked *only* without backticks
* PORT range was different when given as a number or a string

all that is fixed, unused keywords are removed
0803e80ed4fc...
0803e80ed4fcf8da1f8bcccefcb44f448d5d6be9
Sergei Golubchik
MDEV-26910 mysqld_multi starts same instance multiple times with the risk to crash database

Revert the fix 6ce0682b269e and implement deduplication differently -
we want an array here to return groups in the order of appearence.
06cfb8d0c361...
06cfb8d0c36193e0abadb937899b2d249edc52a9
Arcadiy Ivanov
MDEV-40585 Assertion `(data_len == 0) == (data_ptr == ((void *)0))' fails in hp_flush_unaliased_blob_free

`hp_flush_unaliased_blob_free()` asserted that a zero-length blob in the
record buffer carries a `NULL` data pointer.  The SQL layer does not
guarantee that direction of the invariant:

- `Field_blob_compressed::store()` of a zero-length value allocates its
  scratch `String` first and then stores `(length = 0, ptr = value.ptr())`,
  leaving a stale non-`NULL` pointer.  Plain `Field_blob::store()` zeroes
  the whole pack instead, which is why an uncompressed column does not
  reproduce this through SQL.
- `Field_blob::unpack()` points a zero-length blob at the row-based
  replication event buffer, so the applier trips the same assertion on a
  slave-side `HEAP` table with a plain, uncompressed column.

The assertion evaluates only for a column whose old chain was parked for
deferred free, so the failing statement must both park a chain and write
a zero-length blob: `REPLACE` over an existing row, `INSERT ... ON
DUPLICATE KEY UPDATE`, or one replicated row-event group doing the same.
A delete and an insert in separate statements redeem the parking through
the record-less `hp_flush_pending_blob_free_impl()` and are unaffected.

Debug builds only.  Every decision in the engine -- here, in
`hp_write_blobs()` and in `heap_update()` -- tests the stored length and
never the pointer, so release builds store, free and adopt chains
correctly and no wrong data is ever written.

Keep the direction that is guaranteed, a non-empty blob must have a data
pointer, and drop the reverse implication.
BuildBot (0.8.9) working for the MariaDB project.
Page built: Mon 03 Aug 2026 04:44:04 (UTC)