Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
mariadb-PranavTiwari
MDEV-41137 Add CHARACTER SET ANY_CS and CHARSET OF / %CHARSET

Stored routines:

- CHARACTER SET ANY_CS: a routine parameter takes the character set and
  collation of the actual argument at call time, without conversion.
  Allowed on procedure and function parameters only.
- CHARSET OF var (var%CHARSET in sql_mode=ORACLE): a parameter or variable
  takes the character set and collation of a previous parameter or
  variable. Resolved at CREATE time if the source has a fixed character
  set, at call time if it is ANY_CS or anchored to one.
- information_schema.parameters reports NULL character set and collation
  for ANY_CS and call-time anchors.

New tests: main.sp-charset-any_cs, compat/oracle.sp-charset-any_cs.
Vladislav Vaintroub
MDEV-42164 mariadb-import --use-threads can kill its own connection

When a worker fails, kill_tp_connections() sent mysql_kill() for every
worker connection over the failing worker's own connection. It could kill
itself first, and the remaining kills then failed on the dead connection,
so the other workers were not killed and the server kept running their
LOAD DATA after the client exited.

Skip the own connection when killing, and close it afterwards.

Assisted-by: Claude:claude-sonnet-5-5
Oleksandr Byelkin
Fix assertion in Create_tmp_table::finalize lost in 10.11->11.4 merge

Merge 3d174503712 dropped the create_tmp_table_for_derived check in
Item_field::create_tmp_field_from_item_field().

Restore the check.
Vladislav Vaintroub
MDEV-38741 mtr: do not suppress the reason of a client disconnect

The global suppressions of "Aborted connection", in mariadb-test-run.pl
and mtr_warnings.sql, hid the line that the client disconnect watcher
writes when it aborts a command: "Aborted connection N to db: 'x' user:
'u' host: 'h' (Client disconnected)". Nothing showed when the watcher
aborted a connection that it should have left alone. main.locale found
one, because its server log is in German and the English entry did not
match.

Make the two entries skip a line that ends in "(Client disconnected)".
Other aborted connections are suppressed as before.

The tests that drop a client on purpose accept the reason:
have_disconnect_watcher.inc, which the tests of the watcher include,
and dirty_close, loaddata, sp_notembedded, ssl_timeout, mysqltest,
innodb_bug38231, innodb.xa_recovery, mysql_client_test,
mysql_client_test_nonblock, mysql_client_test_comp,
rpl_semi_sync_shutdown_await_ack, spider/bugfix.mdev_35959 (a Spider
link that times out) and mariabackup.nolock_ddl_during_backup_end
(mariabackup ends in the middle of a backup). The Connector/C tests of
the unit suite close connections on purpose, but cannot call
mtr.add_suppression(), so suite.pm adds it before each of them.

A replica closes its connection on purpose when it is stopped, in any
phase of its setup, also while the master runs one of its first
queries. rpl_init.inc accepts the reason for every server of the
replication tests, without writing it to the binlog. A binlog dump
thread logs its disconnect with another message, and only with
log_warnings=4.

Assisted-by: Claude:claude-sonnet-5-5
Vladislav Vaintroub
MDEV-38741 mtr: do not suppress the reason of a client disconnect

The global suppressions of "Aborted connection", in mariadb-test-run.pl
and mtr_warnings.sql, hid the line that the client disconnect watcher
writes when it aborts a command: "Aborted connection N to db: 'x' user:
'u' host: 'h' (Client disconnected)". Nothing showed when the watcher
aborted a connection that it should have left alone. main.locale found
one, because its server log is in German and the English entry did not
match.

Make the two entries skip a line that ends in "(Client disconnected)".
Other aborted connections are suppressed as before.

The tests that drop a client on purpose accept the reason:
have_disconnect_watcher.inc, which the tests of the watcher include,
and dirty_close, loaddata, sp_notembedded, ssl_timeout, mysqltest,
innodb_bug38231, innodb.xa_recovery, mysql_client_test,
mysql_client_test_nonblock, mysql_client_test_comp,
rpl_semi_sync_shutdown_await_ack, spider/bugfix.mdev_35959 (a Spider
link that times out) and mariabackup.nolock_ddl_during_backup_end
(mariabackup ends in the middle of a backup). The Connector/C tests of
the unit suite close connections on purpose, but cannot call
mtr.add_suppression(), so suite.pm adds it before each of them.

A replica closes its connection on purpose when it is stopped, in any
phase of its setup, also while the master runs one of its first
queries. rpl_init.inc accepts the reason for every server of the
replication tests, without writing it to the binlog. A binlog dump
thread logs its disconnect with another message, and only with
log_warnings=4.

Assisted-by: Claude:claude-sonnet-5-5
Oleksandr Byelkin
Merge branch '11.8' into 12.3
Vladislav Vaintroub
MDEV-42164 mariadb-import --use-threads can kill its own connection

When a worker fails, kill_tp_connections() sent mysql_kill() for every
worker connection over the failing worker's own connection. It could kill
itself first, and the remaining kills then failed on the dead connection,
so the other workers were not killed and the server kept running their
LOAD DATA after the client exited.

Skip the own connection when killing, and close it afterwards.

Assisted-by: Claude:claude-sonnet-5-5
Yuchen Pei
MDEV-25848 one range slot per ARRAY index

Before, every MEMBER OF / JSON_OVERLAPS / JSON_CONTAINS point got its
own cloned KEY_PART and key slot, and points were combined only through
index_merge (OR) or by picking one slot (AND).

Now all points on an ARRAY key part share the original slot:

* SEL_ARG_ARRAY holds a list of make_key() points ("the array contains
  any of them") and the summed records_in_range() estimate and cost,
  taken once when the point is built.
* key_or() of two ARRAY ranges takes the union of the points;
  key_and() keeps the operand with the lower estimate, the WHERE
  condition checks the rest.
* KEY_PART cloning (get_claimed_key_part()) is now RTREE-only, so
  ARRAY-only queries size SEL_TREE::keys by param->keys again.
* QUICK_RANGE_SELECT_ARRAY gets one QUICK_RANGE per point and passes
  all points to hlindex_read_first(); json_index reads them in turn
  and skips trefs it has already returned.
* check_quick_select_array() uses the stored estimate, and rejects
  non-ARRAY SEL_ARGs (e.g. IS NULL) on the ARRAY key part.

ORed MEMBER OF and JSON_OVERLAPS now give a single range scan instead
of index_merge sort_union, with no MAX_KEY limit on the number of
points.
forkfun
MDEV-24557 mariadb-dump: import MySQL 5.7+ users, grants, roles

--system=users/all now translates a MySQL 5.7+ source to MariaDB SQL.

- Privileges are mapped; those with no MariaDB equivalent are dropped
  with a "# WARNING". A partial revoke becomes DENY (13.1.1+).
- Translated GRANTs are gated 11.4. All grants are dumped last.
- CREATE USER clauses MariaDB lacks are dropped; an unsupported auth
  plugin locks the account. The hash is read as a string, not as 0x...
  (print_identified_with_as_hex), which MariaDB cannot parse.
- MySQL roles become MariaDB roles, renamed <user>_NNNN if the host is
  not '%' or the name is unavailable (taken, PUBLIC, NONE). Several
  default roles go into <user>_default_roles.
- A mandatory role is granted to PUBLIC, with a "# WARNING": MariaDB
  activates it for every account, MySQL only where it is enabled.
- An unlocked role account (a hybrid) has its privileges on its role,
  which its members get, and on the account itself. The role is
  not its default role, so roles granted to the account stay inactive
  until enabled, as on MySQL. A locked role account has them on its
  role only.
- A comment end in a name is escaped in literals; in a GRANT or DENY the
  statement is emitted ungated, with a "# WARNING".

For every source:
- The /*!80001 CREATE ROLE, DROP ROLE, ALTER USER ... DEFAULT ROLE and
  /*!50701 DROP USER statements are gone: a dump no longer restores
  roles into MySQL.
- The dialect is detected in SQL, not from the version string.
- Roles are created WITH ADMIN mariadb_dump_import_role, so their order
  does not matter.
- DROP USER IF EXISTS is gated /*M!100103, and NO_AUTO_CREATE_USER is
  set while grants run. PUBLIC is not created as a role.

--insert-ignore now overrides --replace, with a warning; together they
gave an invalid REPLACE IGNORE.

Tests replay a dump captured from MySQL 26.7.0; recapture it after
changing the MySQL-source code (see mysql_users_setup.sql).

Assisted-By: Claude:claude-sonnet-5-5
Yuchen Pei
MDEV-25848 bound range optimizer work for many ARRAY index points

With one range slot per ARRAY index, the number of points in a
SEL_ARG_ARRAY is no longer bounded by MAX_KEY, and a JSON_OVERLAPS or
JSON_CONTAINS constant with N elements cost:

* N records_in_range() index dives, one per leaf, even when the
  estimate is never used;
* O(N^2) work in key_or_array(), which copied both point lists into a
  new SEL_ARG_ARRAY on every OR and deduplicated with a linear scan.

Now:

* key_or_array() appends the smaller point list to the larger
  SEL_ARG_ARRAY in place, or to a copy of it if it is shared
  (use_count > 0), without sorting or deduplicating.
* SEL_ARG_ARRAY::normalize() sorts and deduplicates the points once,
  when they are needed: estimates, quick select, optimizer trace.
* SEL_ARG_ARRAY::estimate() computes rows and cost lazily, on first
  use (key_and() or check_quick_select_array()). With
  eq_range_index_dive_limit or more points it dives on that many
  points spread over the sorted list and extrapolates.
* JSON_CONTAINS uses at most eq_range_index_dive_limit elements for
  the range, since each AND costs a dive and fewer conjuncts give a
  superset.

Points are now read in sorted order, so the order of rows from an
ARRAY index range scan may differ.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Vladislav Vaintroub
MDEV-38741 mtr: do not suppress the reason of a client disconnect

The global suppressions of "Aborted connection", in mariadb-test-run.pl
and mtr_warnings.sql, hid the line that the client disconnect watcher
writes when it aborts a command: "Aborted connection N to db: 'x' user:
'u' host: 'h' (Client disconnected)". Nothing showed when the watcher
aborted a connection that it should have left alone. main.locale found
one, because its server log is in German and the English entry did not
match.

Make the two entries skip a line that ends in "(Client disconnected)".
Other aborted connections are suppressed as before.

The tests that drop a client on purpose accept the reason:
have_disconnect_watcher.inc, which the tests of the watcher include,
and dirty_close, loaddata, sp_notembedded, ssl_timeout, mysqltest,
innodb_bug38231, innodb.xa_recovery, mysql_client_test,
mysql_client_test_nonblock, mysql_client_test_comp,
rpl_semi_sync_shutdown_await_ack, spider/bugfix.mdev_35959 (a Spider
link that times out) and mariabackup.nolock_ddl_during_backup_end
(mariabackup ends in the middle of a backup). The Connector/C tests of
the unit suite close connections on purpose, but cannot call
mtr.add_suppression(), so suite.pm adds it before each of them.

A replica closes its connection on purpose when it is stopped, in any
phase of its setup, also while the master runs one of its first
queries. rpl_init.inc accepts the reason for every server of the
replication tests, without writing it to the binlog. A binlog dump
thread logs its disconnect with another message, and only with
log_warnings=4.

Assisted-by: Claude:claude-sonnet-5-5
Vladislav Vaintroub
MDEV-38741 Abort the running statement when its client disconnects

The server notices that a client is gone only when it reads from or
writes to the socket. A statement that waits, for example for an InnoDB
row lock, keeps running, and its transaction keeps its locks.

Watch the client socket from the end of login until the connection ends
or the client sends COM_QUIT, with epoll (Linux), kqueue (BSD, macOS),
or on Windows with WSAEventSelect and an I/O completion port, and the
exit of the client process for a named pipe. When the client has gone,
the watcher sets DISCONNECT_SIGNALED in THD::disconnect_flags. If a
command that expects an answer is running, it is aborted with
KILL_CONNECTION. If the connection is idle, the next such command is
aborted when it starts. COM_QUIT and COM_STMT_CLOSE are not aborted, a
client can send them and then quit.

The events of a round are served by one scan of the connections, under
one read lock of the thread list. A scan for each event took the lock
for a long time: with 256 clients that connect and disconnect, 1300
scans per second cost 17% of the connection rate.

A command whose last reply is being sent is not aborted, there is
nothing left to interrupt. Protocol::end_statement() sets
DISCONNECT_UNINTERRUPTIBLE when it sends the last reply of a command:
an error, or a reply with no more results to follow. A read-only SELECT
sends its result early, from select_send::send_eof(), and the client
can quit while the server still cleans up. COM_STATISTICS and
COM_STMT_PREPARE write their replies themselves and set the flag before
they do. The lost connection is then logged as usual. The watcher and
the connection thread order their accesses with atomic read-modify-write
operations on the flags, no lock is taken for a command.

A connection that is already being killed is left alone. KILL
CONNECTION shuts the socket of its victim, which the watcher reports,
and a second awake of the dying thread would wait for it in
abort_current_cond_wait() for up to 2 seconds.

The socket is not shut down, the connection ends after the error is
sent, and the reason, "Client disconnected", is logged once. A
binlog dump thread logs "BINLOG DUMP client disconnected", and only
with log_warnings=4: a replica closes the connection on purpose when it
stops. A connection with the disconnect noted, idle or only finishing
its reply, shows that State in the processlist. Galera connections are
not aborted.

The global variable disconnect_watcher, ON by default, can be set OFF,
which also ends the watches that exist. It exists only where there is a
watcher: Linux, Windows, the BSDs and macOS, not the embedded server.

New tests: main.disconnect_*, sys_vars.disconnect_watcher_basic,
rpl.rpl_dump_thread_disconnect and mysql_client_test
test_disconnect_*. Some rpl and innodb tests are adjusted to pass with
the watcher ON and OFF.

Assisted-by: Claude:claude-sonnet-5-5
Oleksandr Byelkin
Merge branch 'br-11.8-merge' into br-12.3-merge
Sergei Petrunia
MDEV-38274: Unexpectedly large pages_read_count in main.analyze_stmt_prefetch_count

The testcase checked the values of pages_prefetch_read_count and
pages_read_count that InnoDB reported for the query.

The numbers depend on how fast the prefetch thread is in comparison with
the query thread. It can also happen that we request to prefetch a page
but then still have to wait for it to be read, which is counted in
pages_read_count.

Replace the checks in the file with ones that will hold in all
circumstances.
Vladislav Vaintroub
MDEV-38741 mtr: do not suppress the reason of a client disconnect

The global suppressions of "Aborted connection", in mariadb-test-run.pl
and mtr_warnings.sql, hid the line that the client disconnect watcher
writes when it aborts a command: "Aborted connection N to db: 'x' user:
'u' host: 'h' (Client disconnected)". Nothing showed when the watcher
aborted a connection that it should have left alone. main.locale found
one, because its server log is in German and the English entry did not
match.

Make the two entries skip a line that ends in "(Client disconnected)".
Other aborted connections are suppressed as before.

The tests that drop a client on purpose accept the reason:
have_disconnect_watcher.inc, which the tests of the watcher include,
and dirty_close, loaddata, sp_notembedded, ssl_timeout, mysqltest,
innodb_bug38231, innodb.xa_recovery, mysql_client_test,
mysql_client_test_nonblock, mysql_client_test_comp,
rpl_semi_sync_shutdown_await_ack, spider/bugfix.mdev_35959 (a Spider
link that times out), main.mysqldump (mariadb-import can leave a LOAD DATA
running when it fails, MDEV-42164) and
mariabackup.nolock_ddl_during_backup_end (mariabackup ends in the middle
of a backup). The Connector/C tests of the unit suite close connections
on purpose, but cannot call mtr.add_suppression(), so suite.pm adds it
before each of them.

A replica closes its connection on purpose when it is stopped, in any
phase of its setup, also while the master runs one of its first
queries. rpl_init.inc accepts the reason for every server of the
replication tests, without writing it to the binlog. A binlog dump
thread logs its disconnect with another message, and only with
log_warnings=4.

Assisted-by: Claude:claude-sonnet-5-5
Vladislav Vaintroub
MDEV-38741 Abort the running statement when its client disconnects

The server notices that a client is gone only when it reads from or
writes to the socket. A statement that waits, for example for an InnoDB
row lock, keeps running, and its transaction keeps its locks.

Watch the client socket from the end of login until the connection ends
or the client sends COM_QUIT, with epoll (Linux), kqueue (BSD, macOS),
or on Windows with WSAEventSelect and an I/O completion port, and the
exit of the client process for a named pipe. When the client has gone,
the watcher sets DISCONNECT_SIGNALED in THD::disconnect_flags. If a
command that expects an answer is running, it is aborted with
KILL_CONNECTION. If the connection is idle, the next such command is
aborted when it starts. COM_QUIT and COM_STMT_CLOSE are not aborted, a
client can send them and then quit.

The events of a round are served by one scan of the connections, under
one read lock of the thread list. A scan for each event took the lock
for a long time: with 256 clients that connect and disconnect, 1300
scans per second cost 17% of the connection rate.

A command whose last reply is being sent is not aborted, there is
nothing left to interrupt. Protocol::end_statement() sets
DISCONNECT_UNINTERRUPTIBLE when it sends the last reply of a command:
an error, or a reply with no more results to follow. A read-only SELECT
sends its result early, from select_send::send_eof(), and the client
can quit while the server still cleans up. COM_STATISTICS and
COM_STMT_PREPARE write their replies themselves and set the flag before
they do. The lost connection is then logged as usual. The watcher and
the connection thread order their accesses with atomic read-modify-write
operations on the flags, no lock is taken for a command.

A connection that is already being killed is left alone. KILL
CONNECTION shuts the socket of its victim, which the watcher reports,
and a second awake of the dying thread would wait for it in
abort_current_cond_wait() for up to 2 seconds.

The socket is not shut down, the connection ends after the error is
sent, and the reason, "Client disconnected", is logged once. A
binlog dump thread logs "BINLOG DUMP client disconnected", and only
with log_warnings=4: a replica closes the connection on purpose when it
stops. A connection with the disconnect noted, idle or only finishing
its reply, shows that State in the processlist. Galera connections are
not aborted.

The global variable disconnect_watcher, ON by default, can be set OFF,
which also ends the watches that exist. It exists only where there is a
watcher: Linux, Windows, the BSDs and macOS, not the embedded server.

New tests: main.disconnect_*, sys_vars.disconnect_watcher_basic,
rpl.rpl_dump_thread_disconnect and mysql_client_test
test_disconnect_*. Some rpl and innodb tests are adjusted to pass with
the watcher ON and OFF.

Assisted-by: Claude:claude-sonnet-5-5
Vladislav Vaintroub
MDEV-38741 Abort the running statement when its client disconnects

The server notices that a client is gone only when it reads from or
writes to the socket. A statement that waits, for example for an InnoDB
row lock, keeps running, and its transaction keeps its locks.

Watch the client socket from the end of login until the connection ends
or the client sends COM_QUIT, with epoll (Linux), kqueue (BSD, macOS),
or on Windows with WSAEventSelect and an I/O completion port, and the
exit of the client process for a named pipe. When the client has gone,
the watcher sets DISCONNECT_SIGNALED in THD::disconnect_flags. If a
command that expects an answer is running, it is aborted with
KILL_CONNECTION. If the connection is idle, the next such command is
aborted when it starts. COM_QUIT and COM_STMT_CLOSE are not aborted, a
client can send them and then quit.

The events of a round are served by one scan of the connections, under
one read lock of the thread list. A scan for each event took the lock
for a long time: with 256 clients that connect and disconnect, 1300
scans per second cost 17% of the connection rate.

A command whose last reply is being sent is not aborted, there is
nothing left to interrupt. Protocol::end_statement() sets
DISCONNECT_UNINTERRUPTIBLE when it sends the last reply of a command:
an error, or a reply with no more results to follow. A read-only SELECT
sends its result early, from select_send::send_eof(), and the client
can quit while the server still cleans up. COM_STATISTICS and
COM_STMT_PREPARE write their replies themselves and set the flag before
they do. The lost connection is then logged as usual. The watcher and
the connection thread order their accesses with atomic read-modify-write
operations on the flags, no lock is taken for a command.

A connection that is already being killed is left alone. KILL
CONNECTION shuts the socket of its victim, which the watcher reports,
and a second awake of the dying thread would wait for it in
abort_current_cond_wait() for up to 2 seconds.

The socket is not shut down, the connection ends after the error is
sent, and the reason, "Client disconnected", is logged once. A
binlog dump thread logs "BINLOG DUMP client disconnected", and only
with log_warnings=4: a replica closes the connection on purpose when it
stops. A connection with the disconnect noted, idle or only finishing
its reply, shows that State in the processlist. Galera connections are
not aborted.

The global variable disconnect_watcher, ON by default, can be set OFF,
which also ends the watches that exist. It exists only where there is a
watcher: Linux, Windows, the BSDs and macOS, not the embedded server.

New tests: main.disconnect_*, sys_vars.disconnect_watcher_basic,
rpl.rpl_dump_thread_disconnect and mysql_client_test
test_disconnect_*. Some rpl and innodb tests are adjusted to pass with
the watcher ON and OFF.

Assisted-by: Claude:claude-sonnet-5-5
Sergei Golubchik
skip columnstore better

followup for 550444c3bd4
Yuchen Pei
MDEV-25848 one range slot per ARRAY index

Before, every MEMBER OF / JSON_OVERLAPS / JSON_CONTAINS point got its
own cloned KEY_PART and key slot, and points were combined only through
index_merge (OR) or by picking one slot (AND).

Now all points on an ARRAY key part share the original slot:

* SEL_ARG_ARRAY holds a list of make_key() points ("the array contains
  any of them") and the summed records_in_range() estimate and cost,
  taken once when the point is built.
* key_or() of two ARRAY ranges takes the union of the points;
  key_and() keeps the operand with the lower estimate, the WHERE
  condition checks the rest.
* KEY_PART cloning (get_claimed_key_part()) is now RTREE-only, so
  ARRAY-only queries size SEL_TREE::keys by param->keys again.
* QUICK_RANGE_SELECT_ARRAY gets one QUICK_RANGE per point and passes
  all points to hlindex_read_first(); json_index reads them in turn
  and skips trefs it has already returned.
* check_quick_select_array() uses the stored estimate, and rejects
  non-ARRAY SEL_ARGs (e.g. IS NULL) on the ARRAY key part.

ORed MEMBER OF and JSON_OVERLAPS now give a single range scan instead
of index_merge sort_union, with no MAX_KEY limit on the number of
points.
Oleksandr Byelkin
Merge branch 'bb-11.4-release' into br-11.8-merge
Vladislav Vaintroub
MDEV-42164 mariadb-import --use-threads can kill its own connection

When a worker fails, kill_tp_connections() sent mysql_kill() for every
worker connection over the failing worker's own connection. It could kill
itself first, and the remaining kills then failed on the dead connection,
so the other workers were not killed and the server kept running their
LOAD DATA after the client exited.

Skip the own connection when killing, and close it afterwards.
The test counts the KILL commands the server receives.

Assisted-by: Claude:claude-sonnet-5-5
Sergei Golubchik
skip columnstore better

followup for 550444c3bd4
Oleksandr Byelkin
Merge branch 'bb-11.4-merge' into br-11.8-merge
Yuchen Pei
MDEV-25848 bound range optimizer work for many ARRAY index points

With one range slot per ARRAY index, the number of points in a
SEL_ARG_ARRAY is no longer bounded by MAX_KEY, and a JSON_OVERLAPS or
JSON_CONTAINS constant with N elements cost (see the TODOs):

* N records_in_range() index dives, one per leaf, even when the
  estimate is never used;
* O(N^2) work in key_or_array(), which copied both point lists into a
  new SEL_ARG_ARRAY on every OR and deduplicated with a linear scan.

Now:

* key_or_array() appends the smaller point list to the larger
  SEL_ARG_ARRAY in place, or to a copy of it if it is shared
  (use_count > 0), without sorting or deduplicating.
* SEL_ARG_ARRAY::normalize() sorts and deduplicates the points once,
  when they are needed: estimates, quick select, optimizer trace.
* SEL_ARG_ARRAY::estimate() computes rows and cost lazily, on first
  use (key_and() or check_quick_select_array()). With
  eq_range_index_dive_limit or more points it dives on that many
  points spread over the sorted list and extrapolates.
* JSON_CONTAINS skips duplicate elements of the constant and uses at
  most eq_range_index_dive_limit distinct ones for the range, since
  each AND costs a dive and fewer conjuncts give a superset.

Points are now read in sorted order, so the order of rows from an
ARRAY index range scan may differ.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Yuchen Pei
MDEV-25848 one range slot per ARRAY index

Before, every MEMBER OF / JSON_OVERLAPS / JSON_CONTAINS point got its
own cloned KEY_PART and key slot, and points were combined only through
index_merge (OR) or by picking one slot (AND).

Now all points on an ARRAY key part share the original slot:

* SEL_ARG_ARRAY holds a list of make_key() points ("the array contains
  any of them") and the summed records_in_range() estimate and cost,
  taken once when the point is built.
* key_or() of two ARRAY ranges takes the union of the points;
  key_and() keeps the operand with the lower estimate, the WHERE
  condition checks the rest.
* KEY_PART cloning (get_claimed_key_part()) is now RTREE-only, so
  ARRAY-only queries size SEL_TREE::keys by param->keys again.
* QUICK_RANGE_SELECT_ARRAY gets one QUICK_RANGE per point and passes
  all points to hlindex_read_first(); json_index reads them in turn
  and skips trefs it has already returned.
* check_quick_select_array() uses the stored estimate, and rejects
  non-ARRAY SEL_ARGs (e.g. IS NULL) on the ARRAY key part.

ORed MEMBER OF and JSON_OVERLAPS now give a single range scan instead
of index_merge sort_union, with no MAX_KEY limit on the number of
points.
Vladislav Vaintroub
MDEV-38741 Abort the running statement when its client disconnects

The server notices that a client is gone only when it reads from or
writes to the socket. A statement that waits, for example for an InnoDB
row lock, keeps running, and its transaction keeps its locks.

Watch the client socket from the end of login until the connection ends
or the client sends COM_QUIT, with epoll (Linux), kqueue (BSD, macOS),
or on Windows with WSAEventSelect and an I/O completion port, and the
exit of the client process for a named pipe. When the client has gone,
the watcher sets DISCONNECT_SIGNALED in THD::disconnect_flags. If a
command that expects an answer is running, it is aborted with
KILL_CONNECTION. If the connection is idle, the next such command is
aborted when it starts. COM_QUIT and COM_STMT_CLOSE are not aborted, a
client can send them and then quit.

The events of a round are served by one scan of the connections, under
one read lock of the thread list. A scan for each event took the lock
for a long time: with 256 clients that connect and disconnect, 1300
scans per second cost 17% of the connection rate.

A command whose last reply is being sent is not aborted, there is
nothing left to interrupt. Protocol::end_statement() sets
DISCONNECT_UNINTERRUPTIBLE when it sends the last reply of a command:
an error, or a reply with no more results to follow. A read-only SELECT
sends its result early, from select_send::send_eof(), and the client
can quit while the server still cleans up. COM_STATISTICS and
COM_STMT_PREPARE write their replies themselves and set the flag before
they do. The lost connection is then logged as usual. The watcher and
the connection thread order their accesses with atomic read-modify-write
operations on the flags, no lock is taken for a command.

A connection that is already being killed is left alone. KILL
CONNECTION shuts the socket of its victim, which the watcher reports,
and a second awake of the dying thread would wait for it in
abort_current_cond_wait() for up to 2 seconds.

The socket is not shut down, the connection ends after the error is
sent, and the reason, "Client disconnected", is logged once. A
binlog dump thread logs "BINLOG DUMP client disconnected", and only
with log_warnings=4: a replica closes the connection on purpose when it
stops. A connection with the disconnect noted, idle or only finishing
its reply, shows that State in the processlist. Galera connections are
not aborted.

The global variable disconnect_watcher, ON by default, can be set OFF,
which also ends the watches that exist. It exists only where there is a
watcher: Linux, Windows, the BSDs and macOS, not the embedded server.

New tests: main.disconnect_*, sys_vars.disconnect_watcher_basic,
rpl.rpl_dump_thread_disconnect and mysql_client_test
test_disconnect_*. Some rpl and innodb tests are adjusted to pass with
the watcher ON and OFF.

Assisted-by: Claude:claude-sonnet-5-5
Vladislav Vaintroub
MDEV-42164 mariadb-import --use-threads can kill its own connection

When a worker fails, kill_tp_connections() sent mysql_kill() for every
worker connection over the failing worker's own connection. It could kill
itself first, and the remaining kills then failed on the dead connection,
so the other workers were not killed and the server kept running their
LOAD DATA after the client exited.

Skip the own connection when killing, and close it afterwards.
The test counts the KILL commands the server receives.

Assisted-by: Claude:claude-sonnet-5-5
chanztuying
MDEV-40827: prefetch MHNSW neighbours during search

Issue prefetches for unseen neighbour nodes before evaluating their distances. This overlaps later memory loads with distance calculations for earlier lanes without changing the search result.

On a fixed 200k by 1024-dimensional cosine graph with M=6, a crossed AB/BA warm-cache run (30 paired observations) improved paired median QPS by 10.1% at ef_search=40 (95% CI 8.0%-14.4%) and 13.8% at ef_search=160 (95% CI 11.8%-20.0%), with identical exact-recall means.
mariadb-PranavTiwari
d
Yuchen Pei
MDEV-25848 bound range optimizer work for many ARRAY index points

With one range slot per ARRAY index, the number of points in a
SEL_ARG_ARRAY is no longer bounded by MAX_KEY, and a JSON_OVERLAPS or
JSON_CONTAINS constant with N elements cost (see the TODOs):

* N records_in_range() index dives, one per leaf, even when the
  estimate is never used;
* O(N^2) work in key_or_array(), which copied both point lists into a
  new SEL_ARG_ARRAY on every OR and deduplicated with a linear scan.

Now:

* key_or_array() appends the smaller point list to the larger
  SEL_ARG_ARRAY in place, or to a copy of it if it is shared
  (use_count > 0), without sorting or deduplicating.
* SEL_ARG_ARRAY::normalize() sorts and deduplicates the points once,
  when they are needed: estimates, quick select, optimizer trace.
* SEL_ARG_ARRAY::estimate() computes rows and cost lazily, on first
  use (key_and() or check_quick_select_array()). With
  eq_range_index_dive_limit or more points it dives on that many
  points spread over the sorted list and extrapolates.
* JSON_CONTAINS skips duplicate elements of the constant and uses at
  most eq_range_index_dive_limit distinct ones for the range, since
  each AND costs a dive and fewer conjuncts give a superset.

Points are now read in sorted order, so the order of rows from an
ARRAY index range scan may differ.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Yuchen Pei
MDEV-25848 bound range optimizer work for many ARRAY index points

With one range slot per ARRAY index, the number of points in a
SEL_ARG_ARRAY is no longer bounded by MAX_KEY, and a JSON_OVERLAPS or
JSON_CONTAINS constant with N elements cost:

* N records_in_range() index dives, one per leaf, even when the
  estimate is never used;
* O(N^2) work in key_or_array(), which copied both point lists into a
  new SEL_ARG_ARRAY on every OR and deduplicated with a linear scan.

Now:

* key_or_array() appends the smaller point list to the larger
  SEL_ARG_ARRAY in place, or to a copy of it if it is shared
  (use_count > 0), without sorting or deduplicating.
* SEL_ARG_ARRAY::normalize() sorts and deduplicates the points once,
  when they are needed: estimates, quick select, optimizer trace.
* SEL_ARG_ARRAY::estimate() computes rows and cost lazily, on first
  use (key_and() or check_quick_select_array()). With
  eq_range_index_dive_limit or more points it dives on that many
  points spread over the sorted list and extrapolates.
* JSON_CONTAINS uses at most eq_range_index_dive_limit elements for
  the range, since each AND costs a dive and fewer conjuncts give a
  superset.

Points are now read in sorted order, so the order of rows from an
ARRAY index range scan may differ.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
mariadb-PranavTiwari
Added
chanztuying
MDEV-40827: make the MHNSW prefetch portable to MSVC
Oleksandr Byelkin
Merge branch '10.11' into 11.4
forkfun
MDEV-24557 mariadb-dump: import MySQL 5.7+ users, grants, roles

--system=users/all now translates a MySQL 5.7+ source to MariaDB SQL.

- Privileges are mapped; those with no MariaDB equivalent are dropped
  with a "# WARNING". A partial revoke becomes DENY (13.1.1+).
- Translated GRANTs are gated 11.4. All grants are dumped last.
- CREATE USER clauses MariaDB lacks are dropped; an unsupported auth
  plugin locks the account. The hash is read as a string, not as 0x...
  (print_identified_with_as_hex), which MariaDB cannot parse.
- MySQL roles become MariaDB roles, renamed <user>_NNNN if the host is
  not '%' or the name is unavailable (taken, PUBLIC, NONE). Several
  default roles go into <user>_default_roles.
- A mandatory role is granted to PUBLIC, with a "# WARNING": MariaDB
  activates it for every account, MySQL only where it is enabled.
- An unlocked role account (a hybrid) has its privileges on its role,
  which its members get, and on the account itself. The role is
  not its default role, so roles granted to the account stay inactive
  until enabled, as on MySQL. A locked role account has them on its
  role only.
- A comment end in a name is escaped in literals; in a GRANT or DENY the
  statement is emitted ungated, with a "# WARNING".

For every source:
- The /*!80001 CREATE ROLE, DROP ROLE, ALTER USER ... DEFAULT ROLE and
  /*!50701 DROP USER statements are gone: a dump no longer restores
  roles into MySQL.
- The dialect is detected in SQL, not from the version string.
- Roles are created WITH ADMIN mariadb_dump_import_role, so their order
  does not matter.
- DROP USER IF EXISTS is gated /*M!100103, and NO_AUTO_CREATE_USER is
  set while grants run. PUBLIC is not created as a role.

--insert-ignore now overrides --replace, with a warning; together they
gave an invalid REPLACE IGNORE.

Tests replay a dump captured from MySQL 26.7.0; recapture it after
changing the MySQL-source code (see mysql_users_setup.sql).

Assisted-By: Claude:claude-sonnet-5-5
Oleksandr Byelkin
Merge branch 'br-11.8-merge' into br-12.3-merge
Vladislav Vaintroub
MDEV-38741 mtr: do not suppress the reason of a client disconnect

The global suppressions of "Aborted connection", in mariadb-test-run.pl
and mtr_warnings.sql, hid the line that the client disconnect watcher
writes when it aborts a command: "Aborted connection N to db: 'x' user:
'u' host: 'h' (Client disconnected)". Nothing showed when the watcher
aborted a connection that it should have left alone. main.locale found
one, because its server log is in German and the English entry did not
match.

Make the two entries skip a line that ends in "(Client disconnected)".
Other aborted connections are suppressed as before.

The tests that drop a client on purpose accept the reason:
have_disconnect_watcher.inc, which the tests of the watcher include,
and dirty_close, loaddata, sp_notembedded, ssl_timeout, mysqltest,
innodb_bug38231, innodb.xa_recovery, mysql_client_test,
mysql_client_test_nonblock, mysql_client_test_comp,
rpl_semi_sync_shutdown_await_ack, spider/bugfix.mdev_35959 (a Spider
link that times out) and mariabackup.nolock_ddl_during_backup_end
(mariabackup ends in the middle of a backup). The Connector/C tests of
the unit suite close connections on purpose, but cannot call
mtr.add_suppression(), so suite.pm adds it before each of them.

A replica closes its connection on purpose when it is stopped, in any
phase of its setup, also while the master runs one of its first
queries. rpl_init.inc accepts the reason for every server of the
replication tests, without writing it to the binlog. A binlog dump
thread logs its disconnect with another message, and only with
log_warnings=4.

Assisted-by: Claude:claude-sonnet-5-5
Yuchen Pei
MDEV-25848 one range slot per ARRAY index

Before, every MEMBER OF / JSON_OVERLAPS / JSON_CONTAINS point got its
own cloned KEY_PART and key slot, and points were combined only through
index_merge (OR) or by picking one slot (AND).

Now all points on an ARRAY key part share the original slot:

* SEL_ARG_ARRAY holds a list of make_key() points ("the array contains
  any of them") and the summed records_in_range() estimate and cost,
  taken once when the point is built.
* key_or() of two ARRAY ranges takes the union of the points;
  key_and() keeps the operand with the lower estimate, the WHERE
  condition checks the rest.
* KEY_PART cloning (get_claimed_key_part()) is now RTREE-only, so
  ARRAY-only queries size SEL_TREE::keys by param->keys again.
* QUICK_RANGE_SELECT_ARRAY gets one QUICK_RANGE per point and passes
  all points to hlindex_read_first(); json_index reads them in turn
  and skips trefs it has already returned.
* check_quick_select_array() uses the stored estimate, and rejects
  non-ARRAY SEL_ARGs (e.g. IS NULL) on the ARRAY key part.

ORed MEMBER OF and JSON_OVERLAPS now give a single range scan instead
of index_merge sort_union, with no MAX_KEY limit on the number of
points.
mariadb-PranavTiwari
MDEV-41137 Add CHARACTER SET ANY_CS and CHARSET OF / %CHARSET
Stored routines:

- CHARACTER SET ANY_CS: a routine parameter takes the character set and
  collation of the actual argument at call time, without conversion.
  Allowed on procedure and function parameters only.
- CHARSET OF var (var%CHARSET in sql_mode=ORACLE): a parameter or variable
  takes the character set and collation of a previous parameter or
  variable. Resolved at CREATE time if the source has a fixed character
  set, at call time if it is ANY_CS or anchored to one.
Vladislav Vaintroub
MDEV-42164 mariadb-import --use-threads can kill its own connection

When a worker fails, kill_tp_connections() sent mysql_kill() for every
worker connection over the failing worker's own connection. It could kill
itself first, and the remaining kills then failed on the dead connection,
so the other workers were not killed and the server kept running their
LOAD DATA after the client exited.

Skip the own connection when killing, and close it afterwards.

Assisted-by: Claude:claude-sonnet-5-5