Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Rex Johnston
PQ:  Code tidy ups and fixups of tidyups.

parallel_query_aggregate.test: fix statistics
impersonalize language in can_parallel_scan_jointab_access()
remove dead code (trace_parallel_scan_options)
add an assert to ensure we do not use Item_sum_avg without two packed fields
minor comment fixups
remove enum element NESTED_LOOP_DECLINED and use a local bool instead
fix silly code in pwt_row_layout::build_aggregates (throwing away a pointer
  then searching for it again later)
remove pwt_batch* worker->manager streaming functions
tidy up Parallel_scan_partitioner::Scan_ctx::create_chunks to use
  create_chunks_unsplit
Yuchen Pei
MDEV-40168 Add a per-index argument to the fulltext parser interface

Patch 2/N of Approach 1TB

A fulltext parser plugin is handed one document or query at a time and
told nothing about the index it is serving. Everything in
MYSQL_FTPARSER_PARAM describes the text at hand: the callbacks, the
charset, the buffer, the mode. Two indexes that name the same parser are
indistinguishable to it, so a parser cannot be told what to do per index
-- which path to follow into a document, say, or what to encode the
result as.

Add MYSQL_FTPARSER_PARAM::ftparser_arg for that. The server sets
KEY::ftparser_arg when it opens the table, the storage engine carries it
across to the parser unchanged, and the parser reads it. It is opaque to
the engines: whoever sets it owns it and keeps it alive while the table
is open.

The field is appended to the end of the structure and the interface
version goes to 0x0101, with MYSQL_FTPARSER_MIN_INTERFACE_VERSION left
at 0x0100 so that a parser built against the old header still loads. It
reads the fields it knows at the offsets it knows and never looks at the
new one. Nothing sets ftparser_arg yet, so every parser sees NULL and
nothing changes for any of them.

MyISAM and Aria carry it in MI_KEYDEF / MARIA_KEYDEF beside the parser
itself, copied out of the server's KEY when the table is opened, and
into the parser param in their ftparser_call_initializer().

For InnoDB, its FTS carries the parser down as a function argument --
from dict_index_t into the tokenize and the query paths, one call at a
time -- so the argument beside it travels the same way:

  dict_index_t::ftparser_arg, set from KEY::ftparser_arg when the table
  is opened, next to index->parser

  fts_doc_t::ftparser_arg and fts_query_t::ftparser_arg, next to the
  parser each already carries, plus fts_phrase_t::ftparser_arg for the
  phrase match

  a parameter on fts_tokenize_document(), fts_tokenize_document_next(),
  fts_tokenize_by_parser(), row_merge_fts_doc_tokenize_by_parser(),
  fts_query_match_phrase_terms_by_parser(), fts_query_match_document()
  and fts_parse_by_parser()

so that all four places that build a MYSQL_FTPARSER_PARAM -- the index
build, the DML tokenizer, the boolean query parse and the phrase match
-- set param.ftparser_arg.

fts_query_match_document() declares MY_ATTRIBUTE((nonnull(...))) by
position, and `found' moved from the sixth parameter to the seventh, so
the attribute moves with it. The new argument is NULL for every index
that does not ask for one, which is all of them so far, and must not be
covered by nonnull.

Nothing sets KEY::ftparser_arg yet, so this changes no behaviour: every
parser still sees NULL.

Co-Authored-By: Claude Opus 5 <[email protected]>
Dave Gosselin
MDEV-33616:  Take the read lock many times in perfschema.func_mutex

The wait timer can have a granularity coarser than the time an
uncontended read lock is held, so the recorded duration of one lock can
be zero, which reads back as NULL.  This can cause the test to fail with
a false negative.

Take the lock twenty more times at each measurement point, with the
extra statements silent so the recorded result does not change.  The
mutex part of the test already works this way, since one SELECT
produces ten THR_LOCK::mutex events.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Yuchen Pei
MDEV-40168 Write the multi-valued index definition to the FRM

Patch 3/N of Approach 1TB

A multi-valued index is defined by three things: the base column, the
path into it, and the type its elements are cast to. Today all three live
in one place -- the expression of the internal column the key is over --
so the definition exists only as long as that column does. 1TB drops the
column and makes the base column the key part, which leaves nowhere for
the path and the cast type to live.

Give them a place: EXTRA2_MVI_SPEC, a new extra2 section holding, per
multi-valued index, the printed MVI_ENCODE() call it was declared with.
The printed form rather than anything packed, so there is no binary
format to freeze -- what reads it back is the parser that already reads a
vcol expression. Sparse, one entry per multi-valued index rather than per
key, because most tables have none and then the section is not written at
all.

The section code is 132, above EXTRA2_ENGINE_IMPORTANT. That matters:
strip the section from a 1TB table and the key reads back as a plain
FULLTEXT key over the base column, which is a valid definition meaning
something else entirely. A server that does not know the section has to
refuse the table rather than misread it, and being above the threshold is
what makes it refuse -- see read_extra2().

Nothing consumes the section yet. It is written alongside the internal
column that still carries the same expression, and open_table_from_share()
checks the two against each other under DBUG, so they cannot drift apart
while both exist. That check is what tests this patch: every existing
multi-valued index test exercises it on a debug build.

Co-Authored-By: Claude Opus 5 <[email protected]>
Rucha Deodhar
MDEV-40124: Assertion `m_sp == m_thd->spcont->m_sp' failed virtual
Item *Item_splocal::this_item(), UBSAN : member call on null pointer of
type 'Sp_rcontext_handler' in sql/item.cc

Analysis:
Comparing trigger row objects (IF NEW = OLD) calls cmp_row_type(),
which uses element_index(i). Item_trigger_row was missing its own
element_index() override, falling back to Item_splocal's version
and crashing/asserting on local context lookup.

Fix:
Override element_index(i) in Item_trigger_row to return
m_fields.elem(i) directly.
Marko Mäkelä
MDEV-41021 SET GLOBAL innodb_log_archive ignores read-only mode

log_t::set_archive(): Prevent a crash in log_t::header_rewrite()
by refusing the operation if the log is read-only.
Oleksandr Byelkin
new version of main (3.2) part 2
Rucha Deodhar
MDEV-40127: Server hangs when setting NEW=OLD on a multi-row table

Analysis:
m_fields was keeping old fields around across executions instead of
being cleared, causing stale state and hangs/crashes on re-execution.

Fix:
Clear m_fields in cleanup().
Alexey (Holyfoot) Botchkov
MDEV-36261 XMLTYPE: methods - step 1

Method functions added to the XMLTYPE.
Marko Mäkelä
MDEV-40410: Tight innodb_buffer_pool_size_max on ThreadSanitizer

bur_pool_t::size_in_bytes_max_default: Define as 0 also on
ThreadSanitizer. The symbol __SANITIZE_THREAD__ is predefined
starting with Clang 22 or GCC 7 when building with -fsanitize=thread.
Alexey Yurchenko
MGL-299 Regression in galera_sst_rsync_encrypt_with_key MTR test

Commit b68e29a9c64 explicitly disabled use of SSL encryption in SST
by setting ssl-mode=DISABLED in the top configuration files.
This test is a backward compatibility test so it relies on the
deduction of ssl-mode from the presence of tkey and tcert params
in [sst] section. Unset ssl-mode in config to allow to derive it
from the presence of tkey and tcert.
Dave Gosselin
MDEV-33616:  Skip the redo log upgrade tests without sparse file support

innodb.log_upgrade and innodb.log_upgrade_101_flags build 8GB redo log
files by seeking past the end of an empty file and writing a single
byte.  That needs a filesystem which leaves the skipped range
unallocated.  HFS on macOS allocates every block of it instead, so the
write fails with ENOSPC and the test reports a perl failure.

include/have_sparse_files.inc probes a directory the caller names,
writing one byte 64MB into an empty file there and comparing the
allocated block count against that offset.  Both tests name the vardir
tmp, where they build their redo log files.  The offset stays above 16MB
since APFS allocates the whole range for a file smaller than that rather
than recording a hole.
Dave Gosselin
MDEV-33616:  Match the macOS dlopen error in plugins.multiauth

The client reports why it could not load client_ed25519, and macOS names
every path that dlopen() tried.  Two expressions are added, one for the
chunk that holds the start of that message and one for the chunks that
continue it.

Whether the message arrives in one chunk or several depends on the
vardir, because the path appears four times in the dlopen text.  With
--vardir /Volumes/<repo>/var the line is 417 bytes and fits the 512 byte
buffer that --exec output is read in.  With the default vardir it does
not.

Both expressions stop at a newline.  reg_replace compiles with
REG_DOTALL, so an unrestricted .* runs past the line terminator whenever
the whole message reaches the replacement in one chunk, and the error
line then joins the line after it.
Yuchen Pei
MDEV-40168 Metadata in FRM

WIP note(ycp): the commit message is a concat of two commits that got
squashed.

Patch 3/N of Approach 1TB

1. Write the multi-valued index definition to the FRM

A multi-valued index is defined by three things: the base column, the
path into it, and the type its elements are cast to. Today all three live
in one place -- the expression of the internal column the key is over --
so the definition exists only as long as that column does. 1TB drops the
column and makes the base column the key part, which leaves nowhere for
the path and the cast type to live.

Give them a place: EXTRA2_MVI_SPEC, a new extra2 section holding, per
multi-valued index, the printed MVI_ENCODE() call it was declared with.
The printed form rather than anything packed, so there is no binary
format to freeze -- what reads it back is the parser that already reads a
vcol expression. Sparse, one entry per multi-valued index rather than per
key, because most tables have none and then the section is not written at
all.

The section code is 132, above EXTRA2_ENGINE_IMPORTANT. That matters:
strip the section from a 1TB table and the key reads back as a plain
FULLTEXT key over the base column, which is a valid definition meaning
something else entirely. A server that does not know the section has to
refuse the table rather than misread it, and being above the threshold is
what makes it refuse -- see read_extra2().

Nothing consumes the section yet. It is written alongside the internal
column that still carries the same expression, and open_table_from_share()
checks the two against each other under DBUG, so they cannot drift apart
while both exist. That check is what tests this patch: every existing
multi-valued index test exercises it on a debug build.

2. Refuse a table whose multi-valued index definition is missing

The keys of a table and its EXTRA2_MVI_SPEC image have to describe the
same multi-valued indexes. check_mvi_spec() checks both directions at
open, and a table that fails it does not open -- ER_NOT_FORM_FILE, as for
the FRM's other inconsistencies.

The two directions are worth having for different reasons.

An entry no key claims, or one saying something other than the key says,
means the image and the keys disagree and nothing here can tell which of
them is right.

A multi-valued index with no entry is the one that bites later. The entry
is what the definition becomes once the internal column goes away, and a
key without one reads back as a plain fulltext key over the base column
-- a valid definition, meaning something else. That is the separable
metadata this approach trades for, and this is the check that pays for
it: a table we would open as something it is not is a table to refuse.
The previous patch only asserted the first direction, which is why it
took a bootstrap crash rather than a failing test to notice that
mvi_key_spec() had been reading the wrong column and writing no entries
at all.

Reaching either arm needs an FRM this server would not write, so
mvi_spec_image() grows a mvi_skip_spec_image debug trigger to write none,
and multi_valued_index_debug.test uses it.

Co-Authored-By: Claude Opus 5 <[email protected]>
Oleksandr Byelkin
Merge branch '13.1' into 13.2
Oleksandr Byelkin
Merge branch '13.0' into 13.1
Georgi (Joro) Kodinov
MDEV-38144: Re-recorded the test
Yuchen Pei
MDEV-40168 Narrow the multi-valued index expression

Patch 1/N of Approach 1TB

A multi-valued index can be declared over any expression the parser
accepts:

  KEY idx ((CAST(<anything> AS <type> ARRAY)))

That is wider than the index can stay describable as. Narrow it to

  KEY idx ((CAST(<column> -> '<path>' AS <type> ARRAY)))

where the base column is stored in the table and the path is a literal.
The index is then always (column, path, cast type): the keys of a row
come from the bytes of one column, at a place that does not depend on the
row, so they can be produced by reading that column and looking inside
it. A VIRTUAL generated column has no bytes in a row and is refused for
the same reason.

Two checks, because the parser cannot do both. add_mvi_key_part() has the
expression but not the columns, so it checks the shape with
mvi_base_column(). check_mvi_base_column(), called from
init_key_part_spec(), has the columns -- for ALTER TABLE the list
mysql_prepare_alter_table() merged -- and checks that the base column is
one the index can be built from. It re-checks the shape as well: a table
being rebuilt arrives with the expression read back from the FRM rather
than written by the parser.

Only DDL comes through either. A table already on disk opens through
parse_vcol_defs(), which reads the stored MVI_ENCODE() call.

Nothing that was tested stops working: the arrow and a written-out
json_extract() are the same Item_func_json_extract, so what SHOW CREATE
TABLE prints still feeds back in.

Co-Authored-By: Claude Opus 5 <[email protected]>
Yuchen Pei
MDEV-40168 Narrow the multi-valued index expression

Patch 1/N of Approach 1TB

A multi-valued index can be declared over any expression the parser
accepts:

  KEY idx ((CAST(<anything> AS <type> ARRAY)))

That is wider than the index can stay describable as. Narrow it to

  KEY idx ((CAST(<column> -> '<path>' AS <type> ARRAY)))

where the base column is stored in the table and the path is a literal.
The index is then always (column, path, cast type): the keys of a row
come from the bytes of one column, at a place that does not depend on the
row, so they can be produced by reading that column and looking inside
it. A VIRTUAL generated column has no bytes in a row and is refused for
the same reason.

An exception is

  KEY idx ((CAST(<column> AS <type> ARRAY)))

which we accept and desugar to

  KEY idx ((CAST(<column> -> '$' AS <type> ARRAY)))

Two checks, because the parser cannot do both. add_mvi_key_part() has the
expression but not the columns, so it checks the shape with
mvi_base_column(). check_mvi_base_column(), called from
init_key_part_spec(), has the columns -- for ALTER TABLE the list
mysql_prepare_alter_table() merged -- and checks that the base column is
one the index can be built from. It re-checks the shape as well: a table
being rebuilt arrives with the expression read back from the FRM rather
than written by the parser.

Only DDL comes through either. A table already on disk opens through
parse_vcol_defs(), which reads the stored MVI_ENCODE() call.

Nothing that was tested stops working: the arrow and a written-out
json_extract() are the same Item_func_json_extract, so what SHOW CREATE
TABLE prints still feeds back in.

Co-Authored-By: Claude Opus 5 <[email protected]>
Yuchen Pei
MDEV-40168 Write the multi-valued index definition to the FRM

Patch 3/N of Approach 1TB

A multi-valued index is defined by three things: the base column, the
path into it, and the type its elements are cast to. Today all three live
in one place -- the expression of the internal column the key is over --
so the definition exists only as long as that column does. 1TB drops the
column and makes the base column the key part, which leaves nowhere for
the path and the cast type to live.

Give them a place: EXTRA2_MVI_SPEC, a new extra2 section holding, per
multi-valued index, the printed MVI_ENCODE() call it was declared with.
The printed form rather than anything packed, so there is no binary
format to freeze -- what reads it back is the parser that already reads a
vcol expression. Sparse, one entry per multi-valued index rather than per
key, because most tables have none and then the section is not written at
all.

The section code is 132, above EXTRA2_ENGINE_IMPORTANT. That matters:
strip the section from a 1TB table and the key reads back as a plain
FULLTEXT key over the base column, which is a valid definition meaning
something else entirely. A server that does not know the section has to
refuse the table rather than misread it, and being above the threshold is
what makes it refuse -- see read_extra2().

Nothing consumes the section yet. It is written alongside the internal
column that still carries the same expression, and open_table_from_share()
checks the two against each other under DBUG, so they cannot drift apart
while both exist. That check is what tests this patch: every existing
multi-valued index test exercises it on a debug build.

Co-Authored-By: Claude Opus 5 <[email protected]>
Oleksandr Byelkin
Merge branch '13.1' into mariadb-13.1.1
Yuchen Pei
MDEV-40168 Narrow the multi-valued index expression

Patch 1/N of Approach 1TB

A multi-valued index can be declared over any expression the parser
accepts:

  KEY idx ((CAST(<anything> AS <type> ARRAY)))

That is wider than the index can stay describable as. Narrow it to

  KEY idx ((CAST(<column> -> '<path>' AS <type> ARRAY)))

where the base column is stored in the table and the path is a literal.
The index is then always (column, path, cast type): the keys of a row
come from the bytes of one column, at a place that does not depend on the
row, so they can be produced by reading that column and looking inside
it. A VIRTUAL generated column has no bytes in a row and is refused for
the same reason.

Two checks, because the parser cannot do both. add_mvi_key_part() has the
expression but not the columns, so it checks the shape with
mvi_base_column(). check_mvi_base_column(), called from
init_key_part_spec(), has the columns -- for ALTER TABLE the list
mysql_prepare_alter_table() merged -- and checks that the base column is
one the index can be built from. It re-checks the shape as well: a table
being rebuilt arrives with the expression read back from the FRM rather
than written by the parser.

Only DDL comes through either. A table already on disk opens through
parse_vcol_defs(), which reads the stored MVI_ENCODE() call.

Nothing that was tested stops working: the arrow and a written-out
json_extract() are the same Item_func_json_extract, so what SHOW CREATE
TABLE prints still feeds back in.

Co-Authored-By: Claude Opus 5 <[email protected]>

MDEV-40168 Accept a bare column as the multi-valued index expression

Patch 2/N of Approach 1TB

A column that is itself the array to index needs no path to follow into
it, and

  KEY idx ((CAST(j AS <type> ARRAY)))

is how one would write that. The narrowing refused it, because it is not
a `<column> -> '<path>'' expression. But it means the same as

  KEY idx ((CAST(j->'$' AS <type> ARRAY)))

which the narrowing does accept, so desugar it into that rather than
refuse it. mvi_whole_document() builds the path Item the way the parser
builds the one in `j->'$'', so from there on there is a single form: in
the FRM, in what SHOW CREATE TABLE prints, and on the query side.

The query side has to follow, or the index would be built and then never
used for the form most people would write. get_mvi_index() now compares
with mvi_same_document(), which additionally matches a bare column
against that column at path '$'. A path into the document is still a
different expression and does not match.

Nothing is loosened beyond that: the base column of a bare-column index
still has to be one the index can be built from, and every other shape
the narrowing refused is still refused.

Co-Authored-By: Claude Opus 5 <[email protected]>
Oleksandr Byelkin
new version of main (3.2) part 2
Eric
MDEV-38144: update Optional_metadata_fields to use MariaDB types

Currently Optional_metadata_fields has many members that use classes
from the C++ standard library, most notably the use of std::vector and
std::string. This is inconsistent with the coding standards, as MariaDB
defines its own types for arrays/lists/strings. This patch updates
these variables to use existing MariaDB types.

Additionally, the structure of the fields is changed. Instead of having
a separate list to track each metadata field; the structure now exists
at the column level (Column_metadata), and it has individual member
variables to describe its metadata. This allows for more
straightforward memory management, as everything will now be allocated
at once.

There is also a bug fix in this patch. Prior to this refactor, YEAR
columns would not consume their "unsigned" bit flag, resulting in
numeric columns after a YEAR type to have incorrect signage. This is
now fixed.

Reviewed-by: Brandon Nesterenko <[email protected]>.
Oleksandr Byelkin
new columnstore (df4f261f1f622adb2c266527e3fd3c8c4ddaa636 25.10.7)
Alexey Yurchenko
MGL-299 Regression in galera_sst_rsync_encrypt_with_key MTR test

Commit b68e29a9c64 explicitly disabled use of SSL encryption in SST
by setting ssl-mode=DISABLED in the top configuration files.
This test is a backward compatibility test so it relies on the
deduction of ssl-mode from the presence of tkey and tcert params
in [sst] section. Unset ssl-mode in config to allow to derive it
from the presence of tkey and tcert.
Dave Gosselin
MDEV-33616:  Normalize the strerror text in innodb_fts.index_table

The injected deadlock reaches the client as ER_GET_ERRNO carrying errno
11, and the text comes from my_strerror().  11 is EAGAIN on Linux and
EDEADLK on macOS, so the message reads "Resource temporarily
unavailable" on one and "Resource deadlock avoided" on the other.
Replace the quoted text so the test does not depend on it.
Yuchen Pei
MDEV-40168 Read the array in the mvi fulltext parser

The parser now turns a document into the keys of the index rather than
handing it to the built-in parser: it finds the array the index was
declared over, walks it with Mvi_array_iterator -- the same iterator
MVI_ENCODE and the optimizer use, so that the three cannot disagree about
what a key is -- and hands each encoded element to mysql_add_word().

What an index was declared with arrives in
MYSQL_FTPARSER_PARAM::ftparser_arg as an Mvi_parser_arg: the path to the
array, parsed once, and the datatype its elements are cast to. The parser
cannot allocate -- it runs on the engine's threads during a commit or an
index build -- and it cannot have per-index state of its own either, so
the argument is read-only and everything that changes while a document is
read is on the stack of the parse.

A query stays with the built-in parser. The optimizer encodes the keys
where it builds the query, and boolean mode is the only form it builds, so
MYSQL_FTPARSER_FULL_BOOLEAN_INFO is what tells a query from a document.
The one case that cannot be told apart is a natural-language query, which
comes in the same mode a document being indexed does; it is read as a
document, finds no array, and matches nothing. MATCH ... AGAINST on a
multi-valued index is not something the optimizer writes, and an empty
result is the conservative way for it to go wrong.

A key with nothing in ftparser_arg is not a multi-valued index, only a
fulltext key that happens to name the parser, and keeps behaving exactly
as before.

The argument also has to reach an index that ALTER TABLE builds, which
does not go through the table the server opens afterwards: carry it on
index_def_t next to the parser it belongs with.

No DDL produces such a declaration yet, so a debug keyword plants a fixed
one -- the array at $.tags, encoded as CHAR -- on any fulltext key that
names the parser. That is what the new testcase drives, cross-checking the
keys the index ends up holding against what MVI_ENCODE() makes of the same
array.

Co-Authored-By: Claude Opus 5 <[email protected]>
Yuchen Pei
MDEV-40168 Add the fulltext parser of a multi-valued index

Patch 4/N of Approach 1TB

A multi-valued index is a fulltext index whose tokens are the encoded
elements of a JSON array. Today the server encodes them into a hidden
column and the engine tokenizes that column's text with the built-in
parser. 1TB moves that work into a parser of its own, so that the
elements can be read out of the base column and nothing has to be
materialised.

This is the parser, doing nothing of its own yet. Every mode hands the
text to the built-in parser through param->mysql_parse(), which is what
the engine would have done with no parser at all, so a fulltext index
naming this one behaves exactly as before -- the same rows and the same
tokens in the index, which is what multi_valued_index_parser.test
checks, on all three engines that honour a parser.

What the modes will become is worth saying now, because they are not the
same text:

  MYSQL_FTPARSER_SIMPLE_MODE is a document, the value of the column the
  index is over, and this is where the array will be walked and encoded
  once ftparser_arg carries the path and the datatype. Until then the
  column already holds the keys separated by spaces, which is precisely
  what the built-in parser splits.

  The other two modes are a query -- the boolean string the optimizer
  built, or a phrase being matched -- and those are keys already. There
  will be nothing to encode there even later: the encoding happens where
  the query is built, so that the two sides cannot disagree about what a
  key is.

The plugin is MANDATORY, and named "mvi" because that name goes into the
FRM of every table using it and cannot change afterwards. Its maturity is
STABLE because plugin_add() asserts a mandatory plugin is at least as
mature as the server.

Co-Authored-By: Claude Opus 5 <[email protected]>
Yuchen Pei
Add a per-index argument to the fulltext parser interface

Patch 2/N of Approach 1TB

A fulltext parser plugin is handed one document or query at a time and
told nothing about the index it is serving. Everything in
MYSQL_FTPARSER_PARAM describes the text at hand: the callbacks, the
charset, the buffer, the mode. Two indexes that name the same parser are
indistinguishable to it, so a parser cannot be told what to do per index
-- which path to follow into a document, say, or what to encode the
result as.

Add MYSQL_FTPARSER_PARAM::ftparser_arg for that. The server sets
KEY::ftparser_arg when it opens the table, the storage engine carries it
across to the parser unchanged, and the parser reads it. It is opaque to
the engines: whoever sets it owns it and keeps it alive while the table
is open.

The field is appended to the end of the structure and the interface
version goes to 0x0101, with MYSQL_FTPARSER_MIN_INTERFACE_VERSION left
at 0x0100 so that a parser built against the old header still loads. It
reads the fields it knows at the offsets it knows and never looks at the
new one. Nothing sets ftparser_arg yet, so every parser sees NULL and
nothing changes for any of them.

MyISAM and Aria carry it in MI_KEYDEF / MARIA_KEYDEF beside the parser
itself, copied out of the server's KEY when the table is opened, and
into the parser param in their ftparser_call_initializer().

For InnoDB, its FTS carries the parser down as a function argument --
from dict_index_t into the tokenize and the query paths, one call at a
time -- so the argument beside it travels the same way:

  dict_index_t::ftparser_arg, set from KEY::ftparser_arg when the table
  is opened, next to index->parser

  fts_doc_t::ftparser_arg and fts_query_t::ftparser_arg, next to the
  parser each already carries, plus fts_phrase_t::ftparser_arg for the
  phrase match

  a parameter on fts_tokenize_document(), fts_tokenize_document_next(),
  fts_tokenize_by_parser(), row_merge_fts_doc_tokenize_by_parser(),
  fts_query_match_phrase_terms_by_parser(), fts_query_match_document()
  and fts_parse_by_parser()

so that all four places that build a MYSQL_FTPARSER_PARAM -- the index
build, the DML tokenizer, the boolean query parse and the phrase match
-- set param.ftparser_arg.

fts_query_match_document() declares MY_ATTRIBUTE((nonnull(...))) by
position, and `found' moved from the sixth parameter to the seventh, so
the attribute moves with it. The new argument is NULL for every index
that does not ask for one, which is all of them so far, and must not be
covered by nonnull.

Nothing sets KEY::ftparser_arg yet, so this changes no behaviour: every
parser still sees NULL.

Co-Authored-By: Claude Opus 5 <[email protected]>
Vladislav Vaintroub
MDEV-41080 startup code on Windows, remove checks for existing service

They were not necessary, just try to run as service, and fallback to
command line.

Add some diagnostics - unexpected errors from StartServiceCtrlDispatcher
and RegisterServiceCtrlHandler are now reported to Windows event log.

Also use authoritative service name, returned as first argument
in svc_main by service control manager.
Rex Johnston
PQ: separate finding a range's boundaries from cutting the scan up

add_scan() walked the tree for one range and turned that range's boundaries
into chunks in the same breath. Both halves are per range, and only the first
one should be: whether a chunk may be divided again is a question about the
scan, and a scan may hold many ranges of very different sizes.

So add_scan() now partitions and keeps the boundaries on its Scan_ctx, and the
caller adds every range it has before calling create_chunks() once. Behaviour
is unchanged -- each range still decides for itself, in
Scan_ctx::create_chunks(), exactly as it did before -- and this is the shape
that lets that decision move.

Measured with r_chunks in ANALYZE FORMAT=JSON, the same 50,000 rows of a
million-row table, reached three ways: as one range, as ten equal ranges, and
as one large range with nine tiny ones beside it.

    workers            one range  ten equal  one large + nine tiny
      4    chunks          10        103            28
            pages          458        899          511
      12    chunks          26        432            44
            pages          522      2215          575

Ten ranges cost sixteen times the chunks and four times the pages of one range
holding the same rows. The cost follows the number of ranges big enough to be
partitioned, not how unevenly the rows sit across them, which is why the
lopsided third shape is nearly as cheap as the first. Per-worker row counts are
within the usual 3:1 tail in all three, so the extra chunks buy nothing -- the
pull queue was already balancing the work.

Why the decision has not moved with the call:

It needs a measure of how much work a range holds, and the boundaries do not
carry one. A level-0 walk cuts at the children of the root, so a range narrower
than one of those -- which at this fan-out is any range under about a tenth of
the table, and so most range scans -- yields exactly one boundary whether it
covers ten rows or a hundred thousand. Two rules were built on those counts and
both were worse than what they replaced. Deciding from the total number of
chunks left the third shape's large range unsplit on a single worker while the
other three had ten rows each. Weighting by whole sub-trees, on the reasoning
that a range clamps only the first and last it touches, measured zero nearly
everywhere and serialised all three shapes.

What that rules out is worth keeping: any rule counting boundaries is deciding
without knowing the sizes. The existing per-range rule is in that class too --
it survives because its failure is to over-split, which costs pages rather than
answers. The number that does measure work is the count of sub-ranges a chunk
yields when it is split, which is already computed inside Chunk::split(), where
the piece count is decided.

This commit was prepared with Claude Code (Opus 5), which took the measurements
above, and wrote and discarded both of the rules described.
Dave Gosselin
MDEV-33616:  Charge and credit the same size for the recovery buffer

main.large_pages fails on macOS with "Warning: Memory not freed: 16375"
at shutdown and no accompanying safemalloc report.  The residual stays
at 16375 whether innodb_buffer_pool_size is 8M or 128M, and dropping
--large-pages makes it go away.

recv_sys_t::find_checkpoint() asks for tmp_buf_size, which is
MTR_SIZE_MAX + 9, or 1048585 bytes.  my_large_malloc() rounds that up to
a multiple of the large page size and charges the rounded figure to
global_memory_used, while recv_sys_t::tmp_free() credits back the
1048585 that was requested.  The page size here is 16384, 1048585 rounds
up to 1064960, and the difference is the 16375 reported.  The caller
cannot see the rounded figure because ut_malloc_dontdump() takes the
size by value and, with a null ut_new_pfx_t, has nowhere to report what
it allocated.  ut_malloc_dontdump_size() writes the size back, and
recv_sys_t keeps it in tmp_buf_alloc_size for the free.  tmp_buf_size
remains the capacity that parse() asserts against.

Only macOS rounds up.  my_get_large_page_sizes() has no huge page
interface to consult there, so its fallback branch reports the ordinary
page size as the only large page size and the plain mmap() always
succeeds.  On Linux the candidate is 2 MiB, the MAP_HUGETLB mapping
fails with ENOMEM when no huge pages are reserved, and the retry loop
settles on large_page_size == 0, which records the request unrounded.
No memory was lost either way, since munmap() rounds its length up to a
whole page.  The counter was wrong, and the counter is what MTR checks.

Co-Authored-By: Claude Opus 5 <[email protected]>
Rex Johnston
PQ: divide a multiple-range scan by the work it holds, not a range at a time

Each range of a multiple-range scan divided itself: it compared its own chunk
count against the worker count, found itself short, tagged what it had for
re-splitting, and the splits that followed were per range as well. Ten ranges
did that ten times over and the queue filled with chunks the scan never needed,
without dividing the work any more evenly than a single range does.

So for a scan of more than one range the work is measured first and divided
afterwards. Measuring means walking one level below the sub-trees a level-0
walk finds, which gives a boundary per page: a page is the unit the measure
needs, and the sub-trees are not -- a range narrower than one of those yields a
single boundary whether it holds ten rows or a hundred thousand, which is why
the two rules built on those counts both failed. With every range measured the
same way, the chunks the scan should have -- twice the worker count -- are
shared out in proportion to pages, and each range coalesces its own boundaries
down to its share. Nothing is tagged: the division was made knowing the sizes,
so there is nothing left for a worker to correct at run time.

A scan of one range keeps the division it had. There is only one thing to
divide, it already gets the whole worker count, and leaving it alone keeps this
change off the common path.

Measured with r_chunks in ANALYZE FORMAT=JSON. Ten ranges whose sizes go up by
the golden ratio -- 253, 410, 663, 1073, 1736, 2809, 4545, 7355, 11900, 19255
rows -- against a million-row table. Sizes in that progression because no two
of them are a whole multiple of each other, so nothing can come out balanced by
accident:

    workers      chunks          pages          rows on the busiest worker
              before after    before after        before after
      4        97    18        854  582        14044  14893
      8        169    25      1142  610          7686  8116
      12        216    34      1330  646          5856  5518
      16        261    41      1510  674          4880  4392

Five to six times fewer chunks, a third to a half fewer pages, and the spread
across workers no worse -- better from eight workers up, where the busiest
worker now has less to do than it did. The same ten ranges at equal sizes go
from 103, 223, 432 and 432 chunks to 10, 20, 30 and 50; one large range beside
nine tiny ones goes from 28, 36, 44 and 54 to 18, 26, 34 and 44, with the
per-worker spread between 1.01 and 3.00 throughout. A single range is
unchanged, which is the point of leaving it alone.

Not yet measured, and the reason to: the deeper walk reads the pages one level
down for every range, before the scan starts, where those reads used to happen
inside a re-split and only for the chunks that were tagged. For these range
sets it pays for itself several times over. For a range set much larger than
these it might not, and that is a cost question rather than a correctness one.

This commit was prepared with Claude Code (Opus 5), which took the measurements
above, and wrote and discarded two earlier rules -- one counting chunks, one
counting whole sub-trees -- before finding that neither count measures work.
Rucha Deodhar
MDEV-40127: Server hangs when setting NEW=OLD on a multi-row table

Analysis:
m_fields was keeping old fields around across executions instead of
being cleared, causing stale state and hangs/crashes on re-execution.

Fix:
Clear m_fields in cleanup().
Oleksandr Byelkin
Merge branch 'bb-13.0-release' into main