Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Rex Johnston
MDEV-41179 MAX_KEY_PARTS ranges in select causing SEGV

When a select containing 32 ranges is made on a table containing a
compound key with 32 parts, the range optimizer can run off the end
of a stack variable, invalidly overwriting subsequent stack variables.

In the struct st_sel_arg_range_seq, we have an array
  RANGE_SEQ_ENTRY stack[MAX_REF_PARTS];

MAX_REF_PARTS is 32.

check_quick_select
  / sel_arg_range_seq_init
    initialises stack[0] as NOT a key part
  / sel_arg_range_seq_next
    iterates through the key parts, adding key part n to stack[n+1]

key part #32 gets referenced by step_down_to(), setting seq->i off the
end of the array.

Fix: RANGE_SEQ_ENTRY stack[MAX_REF_PARTS+1];

The above change exposed an issue with key length calculation on
MS Windows.  Calling make_prev_keypart_map(32) caused the resultant
bitmap to be calculated as (1UL << 32) - 1.  Using the MSVC compiler
this resulted in an empty key length calculation during
handler::index_read_map, causing an assertion in ha_innobase::index_read().

As we only need 32 bits to represent our key map, we change the type thus
-typedef ulong key_part_map;
+typedef uint32 key_part_map;

We correct make_keypart_map() and make_prev_keypart_map() to call our
overflow safe my_set_bits().  We also correct bka_range_seq_next()
and bkah_range_seq_next() to use make_prev_keypart_map().

We also add some DBUG_ASSERTS in key_part_map processing elsewhere,
exposing some issues in our BNLH implementation.  We cap the number of
keyuse parts here, altering the explain output of 2 of our tests.
Alexander Barkov
MDEV-41246 "Illegal mix of collations" on the mysql.user view

In progress
Marko Mäkelä
Stream DATA DIRECTORY for ENGINE=InnoDB
Vladislav Vaintroub
MDEV-40955  mysql_client_test needs a resolvable DNS on Linux.

The tests test_proxy_header_connect_errors_reset() and
test_proxy_header_host_denied_not_counted() rely on their client IPs
(192.0.2.x test IPs, per RFC 5737) failing reverse DNS lookup permanently,
which is what a real, working resolver reports for them. Linux's resolver
isn't so RFC-compliant, when it has no route to any nameserver at all: it
reports EAI_AGAIN (temporary) instead, which is deliberately excluded from
connect-error accounting to avoid blocking hosts during a DNS outage.
That silently defeats the max_connect_errors check these tests exercise.

Fix by forcing the deterministic "permanent failure" outcome via the
existing getnameinfo_error_noname debug instrumentation, same as its
sibling tests. Debug-only, like those siblings, since the workaround
needs DBUG_EXECUTE_IF.

Assisted-By: Claude Sonnet 5 <[email protected]>
Yuchen Pei
MDEV-39923 tmp
Rex Johnston
MDEV-41179 MAX_KEY_PARTS ranges in select causing SEGV

When a select containing 32 ranges is made on a table containing a
compound key with 32 parts, the range optimizer can run off the end
of a stack variable, invalidly overwriting subsequent stack variables.

In the struct st_sel_arg_range_seq, we have an array
  RANGE_SEQ_ENTRY stack[MAX_REF_PARTS];

MAX_REF_PARTS is 32.

check_quick_select
  / sel_arg_range_seq_init
    initialises stack[0] as NOT a key part
  / sel_arg_range_seq_next
    iterates through the key parts, adding key part n to stack[n+1]

key part #32 gets referenced by step_down_to(), setting seq->i off the
end of the array.

Fix: RANGE_SEQ_ENTRY stack[MAX_REF_PARTS+1];

The above change exposed an issue with key length calculation on
MS Windows.  Calling make_prev_keypart_map(32) caused the resultant
bitmap to be calculated as (1UL << 32) - 1.  Using the MSVC compiler
this resulted in an empty key length calculation during
handler::index_read_map, causing an assertion in ha_innobase::index_read().

As we only need 32 bits to represent our key map, we change the type thus
-typedef ulong key_part_map;
+typedef uint32 key_part_map;

We correct make_keypart_map() and make_prev_keypart_map() to call our
overflow safe my_set_bits().  We also correct bka_range_seq_next()
and bkah_range_seq_next() to use make_prev_keypart_map().

We also add some DBUG_ASSERTS in key_part_map processing elsewhere,
exposing some issues in our BNLH implementation.  We cap the number of
keyuse parts here, altering the explain output of 2 of our tests.
Yuchen Pei
Ask contributors to re-request review when ready

Update COMMUNITY_CONTRIBUTIONS.md to ask external contributors to
explicitly request a review once they have addressed the points
raised in the current review round and the pull request is ready
for another round of review.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Daniel Black
Merge branch '10.11' into 10.11-MDEV-40955
Vladislav Vaintroub
MDEV-40955  mysql_client_test needs a resolvable DNS on Linux.

The tests test_proxy_header_connect_errors_reset() and
test_proxy_header_host_denied_not_counted() rely on their client IPs
(192.0.2.x test IPs, per RFC 5737) failing reverse DNS lookup permanently,
which is
what a real, working resolver reports for it. Linux's resolver isn't so
RFC-compliant, when it has no route to any nameserver at all: it reports
EAI_AGAIN (temporary) instead, which is deliberately excluded from
connect-error accounting to avoid blocking hosts during a DNS outage.
That silently defeats the max_connect_errors check this test exercises.

Fix by forcing the deterministic "permanent failure" outcome via the
existing getnameinfo_error_noname debug instrumentation, same as its
sibling tests. Debug-only, like those siblings, since the workaround
needs DBUG_EXECUTE_IF.

Assisted-By: Claude Sonnet 5 <[email protected]>
Vladislav Vaintroub
libmariadbd: an embedded server library that starts a private mariadbd

With -DWITH_EMBEDDED_SERVER=ON build libmariadbd (shared and static), the
client library compiled so that

- mysql_server_init() starts a private mariadbd, reachable only through a
  private Unix socket (named pipe on Windows), passing it the arguments
  and option groups of the application;
- mysql_real_connect() to the local host goes to that server;
- mysql_server_end() stops it.

The launcher, ma_embedded_launcher.c, is compiled in only for libmariadbd.
This replaces the hooks (mariadb_set_embedded_hooks) that registered a
launcher at run time: the library is chosen when the application is linked,
as it was with libmysqld. libmariadbd also exports mariadb_embedded_socket()
and mariadb_embedded_error(). The option is WITH_EMBEDDED_SERVER, or
CONC_WITH_EMBEDDED_SERVER when this is a subproject of the server.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Vladislav Vaintroub
MDEV-11111: libmariadbd is chosen when linking, not at run time

Replace the run-time registration of the launcher (hooks in libmariadb,
a glue file, mysql and mysqltest calling mariadb_embedded_register()) by
a build option, as it was with libmysqld.

- WITH_EMBEDDED_SERVER builds libmariadbd in Connector/C (the launcher
  lives there, see its commit). The option is restored, and set for the
  RPM and DEB release builds again.
- mysqltest_embedded is mysqltest linked with libmariadbd; mtr
  --embedded-server runs it, and it starts the private mariadbd for the
  --server-arg it is given. The mysql client does not start a server any
  more, so --server-arg is "not supported" again and main.mysql tests
  that. mysql_client_test_embedded and mariadb-embedded are not built, as
  no test uses them.
- libmariadbd/ keeps only two small client programs that use libmariadbd.
- Packaging lists mysqltest_embedded and libmariadbd.a again.
- sys_vars.version finds an embedded run by the name of mysqltest again.

Tested on Linux and Windows, normal and --embedded-server.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Vladislav Vaintroub
MDEV-11111: mysqltest_embedded waits for the generated error headers

mysqltest includes mysqld_ername.h, which GenError generates. The other
client programs depend on GenError, but mariadb-test-embedded did not, so
it failed to compile when it was built first (buildbot, make -j).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Yuchen Pei
Ask contributors to re-request review when ready

Update COMMUNITY_CONTRIBUTIONS.md to ask external contributors to
explicitly request a review once they have addressed the points
raised in the current review round and the pull request is ready
for another round of review.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Mohammad Tafzeel Shams
MDEV-37467: InnoDB Instant ALTER TABLE is not crash safe

The hidden metadata record of instant ALTER TABLE was not written
crash-safely, and recovery could fail to roll it back. These are
independent problems.

First, the metadata record may include externally stored BLOB
metadata. The existing BLOB storage path in
btr_store_big_rec_extern_fields() writes the clustered index record
first, with zero BLOB pointers, and only fills in the BLOB pointers
afterwards. If the server is killed after the mini-transaction that
wrote the (incomplete) metadata record was durably committed, but
before the BLOB pointers were written, the table could become
inaccessible on recovery.

Make metadata BLOB storage crash-safe by writing the BLOB pages and
computing their pointers before the metadata record itself is
inserted or updated, so that the record is always written with
complete BLOB pointers. If the server is killed before the metadata
record is written, the already-written BLOB pages are merely
orphaned, which is safe.

Second, trx_undo_report_row_operation() writes the undo log record in
a mini-transaction of its own, which is committed before the
mini-transaction that writes the metadata record. Because
innobase_instant_try() had already updated SYS_COLUMNS and SYS_TABLES
in earlier mini-transactions, a kill in between left a durable undo
log record for the table while the metadata record was unchanged. On
recovery, trx_resurrect_table_locks() would then load the table
definition before the incomplete transaction was rolled back. The
data dictionary described the table as it would be after the
operation, while the metadata record still described it as it was
before, and btr_cur_instant_init() failed on that disagreement.

Write the undo log record of the metadata record in the same
mini-transaction that inserts or updates the record, so that the two
cannot be separated by a crash: until that mini-transaction is
committed, neither of them is durable.

An undo log record is never split between pages. If the DEFAULT
values of the columns being added are large enough that the undo log
record for updating the metadata record would not fit on one page,
innobase_instant_try() would fail. Determine this before the operation
starts, so that it can be performed by another algorithm instead.

Third, the table definition that recovery loads need not correspond to
the metadata record. dict_load_table_one() reads the committed version
of the SYS_TABLES record, and escalates to READ UNCOMMITTED only when
it finds a SYS_COLUMNS record that was written by a transaction that is
still active. The number of SYS_COLUMNS records that dict_load_columns()
reads is derived from SYS_TABLES.N_COLS, which was read from the
committed version. The record of a column that the operation appended is
located after that many records, so it is never read and the operation
goes unnoticed. Only an instant ALTER TABLE that merely appends columns
can escape this way: ADD COLUMN ... FIRST, DROP COLUMN and column
reordering rewrite the SYS_COLUMNS records of already existing columns.

Detect this on the SYS_TABLES record itself, which is located by table
name and therefore does not depend on N_COLS. Every instant ALTER TABLE
that changes the columns updates that record, because
innobase_instant_try() invokes innodb_update_cols().

Fourth, the rollback writes a metadata record that comprises fewer
fields than the table definition describes, because
btr_cur_trim_alter_metadata() shortens it to the number of fields that
it comprised before the operation. That number determines the size of
the null flag bitmap, and hence the position of the array of field
lengths. rec_init_offsets_comp_ordinary() derives it from the record,
while the two functions that write the record derived it from the table
definition and asserted that the two agree.

- btr_store_big_rec_metadata():
  New function to store the off-page columns of a metadata record
  ahead of time. Each BLOB page is allocated and linked in its own
  mini-transaction, and the resulting BLOB pointers are written
  directly into the (heap-resident) index entry. On failure, it frees
  any pages it already allocated and resets the pointers to zero.

- btr_free_big_rec_metadata():
  New helper to free the BLOB pages written by
  btr_store_big_rec_metadata() and reset the entry's BLOB pointers
  to zero, used both on failure inside that function and by its
  callers when the metadata record ends up not being written.

- row_ins_clust_index_entry_low():
  For a metadata entry that needs external storage, convert it to a
  big record and call btr_store_big_rec_metadata() (with
  log_free_check() allowed, since no latches are held yet) before
  inserting the record. On failure, free the metadata BLOBs and
  convert the entry back.

- btr_cur_pessimistic_update():
  When updating a metadata record that requires external storage,
  call btr_store_big_rec_metadata() (without log_free_check(),
  since index and page latches are held) before modifying the record,
  and free the temporary big_rec vector via btr_free_big_rec_metadata()
  or dtuple_big_rec_free() on the various failure/success paths.

- btr_cur_optimistic_insert():
  Remove the special-cased jump to convert_big_rec for metadata
  entries, since their BLOBs are now always stored ahead of time by
  the caller; assert that a metadata entry never needs external
  storage at this point.

- innobase_instant_try():
  Since btr_cur_pessimistic_update() now stores metadata BLOBs
  before updating the record, big_rec is always NULL here; assert
  this instead of calling btr_store_big_rec_extern_fields().

- trx_undo_report_row_operation():
  New parameter caller_mtr. If it is specified, the undo log record
  is written in that mini-transaction, which is never committed or
  restarted here. An undo log page is added within the same
  mini-transaction if the record does not fit on the current one. A
  temporary table never uses the caller's mini-transaction, because
  that would require changing its logging mode. All other callers
  pass NULL and are unaffected.
  Encapsulate the parameters that describe the row change
  (clust_entry, update, cmpl_info, rec, offsets) in the new type
  trx_undo_row_op, which of the fields are set depends on the
  operation, which the type documents.

- btr_cur_ins_lock_and_undo(), btr_cur_upd_lock_and_undo():
  For an instant ALTER TABLE metadata record, pass the
  mini-transaction that is going to insert or modify the record.

- trx_undo_max_rec_size():
  New function to determine the maximum size of an undo log record,
  that is, the space available on an empty undo log page.

- ha_innobase::check_if_supported_inplace_alter():
  Refuse ALGORITHM=INSTANT if the metadata record already exists and
  the undo log record for updating it would exceed
  trx_undo_max_rec_size(). trx_undo_page_report_modify() stores the
  DEFAULT value of each column that is being added in that record in
  full, inline. No such limit applies when the metadata record is
  being inserted, because trx_undo_page_report_insert() writes
  TRX_UNDO_INSERT_METADATA and no field data.

- dict_load_table_one():
  If the SYS_TABLES record was written by a transaction that is still
  active, load the table definition as READ UNCOMMITTED. A
  delete-marked record is excluded, because SYS_TABLES.NAME is the
  clustered index key: RENAME TABLE delete-marks the record of the old
  name, and the definition that corresponds to that name is the one
  that precedes the rename.

- dict_sys_tables_rec_read():
  Report whether the current version of the record was written by a
  transaction that has not been committed. The function already
  determines this in order to decide whether to read an older
  version of the record, and used to discard the answer. Store the
  fields that are read in the new type dict_sys_tables_rec, instead
  of in separate output parameters. A caller that does not want a
  delete-marked record to be reported as not found used to indicate
  that by specifying trx_id as nullptr; that is now the parameter
  skip_deleted.

- dict_load_table_low():
  New parameter uncommitted_rec, which is passed on to
  dict_sys_tables_rec_read().

- rec_get_converted_size_comp_prefix_low(),
  rec_convert_dtuple_to_rec_comp():
  For a record that includes a metadata BLOB, determine the number of
  nullable fields from the tuple, by way of
  dict_index_t::get_n_nullable(), and not from
  dict_index_t::n_nullable. This is what
  rec_init_offsets_comp_ordinary() does, and it is equivalent for a
  tuple that comprises all fields of the index. Relax the assertions
  that required the tuple to comprise all of them.

- Added test in innodb.instant_alter and innodb.instant_alter_crash
  to test normal working of INSTANT ALTER, crash safety and full table.
Vladislav Vaintroub
MDEV-40955  mysql_client_test needs a resolvable DNS on Linux.

The tests test_proxy_header_connect_errors_reset() and
test_proxy_header_host_denied_not_counted() rely on their client IPs
(192.0.2.x test IPs, per RFC 5737) failing reverse DNS lookup permanently,
which is what a real, working resolver reports for them. Linux's resolver
isn't so RFC-compliant, when it has no route to any nameserver at all: it
reports EAI_AGAIN (temporary) instead, which is deliberately excluded from
connect-error accounting to avoid blocking hosts during a DNS outage.
That silently defeats the max_connect_errors check these tests exercise.

Fix by forcing the deterministic "permanent failure" outcome via the
existing getnameinfo_error_noname debug instrumentation, same as its
sibling tests. Debug-only, like those siblings, since the workaround
needs DBUG_EXECUTE_IF.

Assisted-By: Claude Sonnet 5 <[email protected]>
Georgi (Joro) Kodinov
Added a reference to the github pull requests docs.
Yuchen Pei
MDEV-39923 tmp
Yuchen Pei
Ask contributors to re-request review when ready

Update COMMUNITY_CONTRIBUTIONS.md to ask external contributors to
explicitly request a review once they have addressed the points
raised in the current review round and the pull request is ready
for another round of review.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Vladislav Vaintroub
MDEV-11111: fix embedded test failures seen on Linux

- The launcher connects once to see that the private server listens. The
  server logged that as an aborted unauthenticated connection, in the
  language of the server, so mtr did not suppress it (main.locale). Do
  not log it in embedded mode.
- sys_vars.version found embedded runs by the name of mysqltest; use the
  --server-arg it is now given.
- sys_vars.port_basic, skip_networking_basic and socket_basic show the
  TCP and socket settings, which the private server sets by itself; skip
  them in embedded runs.

Tested on Linux in normal and --embedded-server mode.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Marko Mäkelä
fixup! f02da92fc678477845bf6f541772975e2d49130e
Alexander Barkov
MDEV-41246 "Illegal mix of collations" on the mysql.user view

This SQL script failed:

SET NAMES latin1 COLLATE latin1_swedish_ci;
CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1;
SET NAMES big5 COLLATE big5_chinese_ci;
SELECT * FROM v1 WHERE c1='y';

with the following error:

ERROR 1267 (HY000): Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE) for operation '='

Note, latin1_swedish_ci and big5_chinese_ci are used here as examples.
The error also happened with different collation combinations.

Fix main idea:

If two collations have equal comparison rules (known as "tailoring")
on a given character repertoire,
like latin1_swedish_ci and big5_chinese_ci on ASCII letters,
then the "Illegal mix of collation" error can be avoided in a comparison
operator. We can choose any of the sides as the operation effective
collation - the result will be equal.

Most important details:

- Splitting enum_repertoire_t into smaller subsets,
  for better repertoire granularity.
  A variable holding a repertoire value can now have multiple
  MY_REPEROIRE_XXX flags set.

  This patch implements detecting tailoring equality on this reperoires:
  * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9].
  * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore
  * MY_REPERTOIRE_ASCII      - the entire range U+0000..U+007F

- Adding a new virtual function "tailoring" in my_collation_handler_st

  It returns the tailoring on the given repertoire for the given collation.

  If cs1->cset->tailoring(cs1, some_repertoire) returns {0,0},
  it means illegal mix optimization cannot be used for this collation
  on the given repertoire.

  If these calls:
    tr1= cs1->cset->tailoring(cs1, some_repertoire);
    tr2= cs2->cset->tailoring(cs2, some_repertoire);
  return both non-NULL results and tr1.ptr==tr2.ptr,
  then these collations are equal on the given repertoire
  and are mutually replaceable for a comparison operator,
  so "Illegal mix of collations" can be avoided.

- Adding a new method DTCollation::aggregate_by_repertoire().

- Adding a new flag MY_COLL_ALLOW_BY_REPERTOIRE.
  It indicates to DTCollation::aggregate() that the illegal
  mix optimization by repertoire can be used in the given context.

  MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_REPERTOIRE.
  Note, only comparison operators pass this flag.
  Functions returning a string result do not pass this flag,
  because in operations like CONCAT(a,b) we still need to evaluate
  precisely the collation of the result - we cannot just choose a collation
  of one of the sides (even if they are compatible on the given repertoire).

- As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of bits
  rather than a single bit, the way how to detect "is only ASCII"
  repertoires has changed in the code.

  For example:
    // repertoire *IS* ascii
    if (repertoire == MY_REPERTOIRE_ASCII)

  has changed in multiple places in the code to

    // repertoire *HAS* only ascii characters
    if (!(repertoire & ~MY_REPERTOIRE_ASCII))

- New flags were added int for CHARSET_INFO::state
  * MY_CS_ASCII_BINARY_CI - for simple 8bit case insensitive collations.
    It means that this collation does not has no irregularities
    on the ASCII range.

  * MY_CS_IDENT_BINARY_CI - for simple 8bit case insensitive collations.
    It means that this collation has not irregularities
    on the IDENT subrange only (but can have irregularities say on
    punctuation).

  * MY_CS_ASCII_STD_UCA - for UCA collations.
    It means that a UCA collation does not reorder ASCII letters.

- strings/conf_to_src.c was modified to detect and print
  MY_CS_ASCII_BINARY_CI and MY_CS_IDENT_BINARY_CI flags.

- strings/ctype-extra.c was regenerated with new flags.

- Adding a number of MTR tests in plugin/func_test/mysql-test/func_test/.
  They display a tailoring by collation name and repertoire as returned by:
    cs->cset->tailoring(cs, some_repertoire)
  A dynamically linked plugin function collation_tailoring() was added
  for the purpose of these tests.

- Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test
  They display two-dimensional charts showing which collations
  are compatible on which repertoires.

- Adding a number of MTR tests in the form of the originally
  reported stript for various collations:

    SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...';
Georgi (Joro) Kodinov
Added a reference to the github pull requests docs.
Rex Johnston
MDEV-41179 MAX_KEY_PARTS ranges in select causing SEGV

When a select containing 32 ranges is made on a table containing a
compound key with 32 parts, the range optimizer can run off the end
of a stack variable, invalidly overwriting subsequent stack variables.

In the struct st_sel_arg_range_seq, we have an array
  RANGE_SEQ_ENTRY stack[MAX_REF_PARTS];

MAX_REF_PARTS is 32.

check_quick_select
  / sel_arg_range_seq_init
    initialises stack[0] as NOT a key part
  / sel_arg_range_seq_next
    iterates through the key parts, adding key part n to stack[n+1]

key part #32 gets referenced by step_down_to(), setting seq->i off the
end of the array.

Fix: RANGE_SEQ_ENTRY stack[MAX_REF_PARTS+1];

The above change exposed an issue with key length calculation on
MS Windows.  Calling make_prev_keypart_map(32) caused the resultant
bitmap to be calculated as (1UL << 32) - 1.  Using the MSVC compiler
this resulted in an empty key length calculation during
handler::index_read_map, causing an assertion in ha_innobase::index_read().

As we only need 32 bits to represent our key map, we change the type thus
-typedef ulong key_part_map;
+typedef uint32 key_part_map;

We correct make_keypart_map() and make_prev_keypart_map() to call our
overflow safe my_set_bits().  We also correct bka_range_seq_next()
and bkah_range_seq_next() to use make_prev_keypart_map().

We also add some DBUG_ASSERTS in key_part_map processing elsewhere,
exposing some issues in our BNLH implementation.  We cap the number of
keyuse parts here, altering the explain output of 2 of our tests.
Yuchen Pei
MDEV-41322 Convert KEY_NOT_FOUND to EOF in "unordered" partition index scans during index_next[_same]/index_prev calls

ha_partition::handle_unordered_scan_next_partition is called in a
variety of accesses, including index_read, index_prev, and index_next.

ha_partition::handle_unordered_next and
ha_partition::handle_unordered_prev are called from index_next and
index_prev accesses. They check for signs of end of
scan (m_part_spec.start_part == NO_CURRENT_PART_ID) and exit early if
that is the case. HA_ERR_END_OF_FILE (EOF) is commonly paired with
assigning NO_CURRENT_PART_ID to m_part_spec.start_part whenever
possible, to signal the end of scan.

The error HA_ERR_KEY_NOT_FOUND means the requested key is not found.
It should not mean the end of scan, when for example
ha_partition::handle_unordered_scan_next_partition is called from
index_read, because a subsequent index_next[_same] / index_prev call
would then incorrectly return immediately from
ha_partition::handle_unordered_next /
ha_partition::handle_unordered_prev. This is why HA_ERR_KEY_NOT_FOUND
is retained and returned in
ha_partition::handle_unordered_scan_next_partition.

But if the call is from index_next[_same] / index_prev,
HA_ERR_KEY_NOT_FOUND should indeed mean end of scan. In this patch, we
ensure this is the case by converting HA_ERR_KEY_NOT_FOUND to EOF in
ha_partition::handle_unordered_next /
ha_partition::handle_unordered_prev.
Yuchen Pei
MDEV-41366 Check prefix key match in partition unordered index scan

The idea of Case 2 in can_skip_merging_scans is that when key prefix
is fixed, and the infix that is also the "partition by range" column,
we can scan each partition in order. This relies on accurately
returning EOF when a partition has no (more) matching rows. It is
possible to have no more matching rows at the first index read of a
partition in a LAST_OR_PREV access, in which case we need to check the
prefix match to rule out false positives and return EOF correctly
Vladislav Vaintroub
MDEV-40955  mysql_client_test needs a resolvable DNS on Linux.

The test test_proxy_header_connect_errors_reset() relies on 192.0.2.50
(test IP, per RFC 5737) failing reverse DNS lookup permanently, which is
what a real, working resolver reports for it. Linux's resolver isn't so
RFC-compliant, when it has no route to any nameserver at all: it reports
EAI_AGAIN (temporary) instead, which is deliberately excluded from
connect-error accounting to avoid blocking hosts during a DNS outage.
That silently defeats the max_connect_errors check this test exercises.

Fix by forcing the deterministic "permanent failure" outcome via the
existing getnameinfo_error_noname debug instrumentation, same as its
sibling tests. Debug-only, like those siblings, since the workaround
needs DBUG_EXECUTE_IF.

Assisted-By: Claude Sonnet 5 <[email protected]>
Brad Smith
crc32c: check elf_aux_info() return value in ppc64 probe

elf_aux_info(3) leaves the output buffer unmodified on failure, so
ignoring the return value could test an uninitialized cpufeatures and
wrongly enable the POWER8 vector-crypto path.

Treat failure as "no features" so the probe falls back to the generic
implementation.
Alexander Barkov
MDEV-41246 "Illegal mix of collations" on the mysql.user view

This SQL script failed:

SET NAMES latin1 COLLATE latin1_swedish_ci;
CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1;
SET NAMES big5 COLLATE big5_chinese_ci;
SELECT * FROM v1 WHERE c1='y';

with the following error:

ERROR 1267 (HY000): Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE) for operation '='

Note, latin1_swedish_ci and big5_chinese_ci are used here as examples.
The error also happened with different collation combinations.

Fix main idea:

If two collations have equal comparison rules (known as "tailoring")
on a given character repertoire,
like latin1_swedish_ci and big5_chinese_ci on ASCII letters,
then the "Illegal mix of collation" error can be avoided in a comparison
operator. We can choose any of the sides as the operation effective
collation - the result will be equal.

Most important details:

- Splitting enum_repertoire_t into smaller subsets,
  for better repertoire granularity.
  A variable holding a repertoire value can now have multiple
  MY_REPEROIRE_XXX flags set.

  This patch implements detecting tailoring equality on this reperoires:
  * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9].
  * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore
  * MY_REPERTOIRE_ASCII      - the entire range U+0000..U+007F

- Adding a new virtual function "tailoring" in my_collation_handler_st

  It returns the tailoring on the given repertoire for the given collation.

  If cs1->cset->tailoring(cs1, some_repertoire) returns {0,0},
  it means illegal mix optimization cannot be used for this collation
  on the given repertoire.

  If these calls:
    tr1= cs1->cset->tailoring(cs1, some_repertoire);
    tr2= cs2->cset->tailoring(cs2, some_repertoire);
  return both non-NULL results and tr1.ptr==tr2.ptr,
  then these collations are equal on the given repertoire
  and are mutually replaceable for a comparison operator,
  so "Illegal mix of collations" can be avoided.

- Adding a new method DTCollation::aggregate_by_repertoire().

- Adding a new flag MY_COLL_ALLOW_BY_REPERTOIRE.
  It indicates to DTCollation::aggregate() that the illegal
  mix optimization by repertoire can be used in the given context.

  MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_REPERTOIRE.
  Note, only comparison operators pass this flag.
  Functions returning a string result do not pass this flag,
  because in operations like CONCAT(a,b) we still need to evaluate
  precisely the collation of the result - we cannot just choose a collation
  of one of the sides (even if they are compatible on the given repertoire).

- As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of bits
  rather than a single bit, the way how to detect "is only ASCII"
  repertoires has changed in the code.

  For example:
    // repertoire *IS* ascii
    if (repertoire == MY_REPERTOIRE_ASCII)

  has changed in multiple places in the code to

    // repertoire *HAS* only ascii characters
    if (!(repertoire & ~MY_REPERTOIRE_ASCII))

- New flags were added int for CHARSET_INFO::state
  * MY_CS_ASCII_BINARY_CI - for simple 8bit case insensitive collations.
    It means that this collation does not has no irregularities
    on the ASCII range.

  * MY_CS_IDENT_BINARY_CI - for simple 8bit case insensitive collations.
    It means that this collation has not irregularities
    on the IDENT subrange only (but can have irregularities say on
    punctuation).

  * MY_CS_ASCII_STD_UCA - for UCA collations.
    It means that a UCA collation does not reorder ASCII letters.

- strings/conf_to_src.c was modified to detect and print
  MY_CS_ASCII_BINARY_CI and MY_CS_IDENT_BINARY_CI flags.

- strings/ctype-extra.c was regenerated with new flags.

- Adding a number of MTR tests in plugin/func_test/mysql-test/func_test/.
  They display a tailoring by collation name and repertoire as returned by:
    cs->cset->tailoring(cs, some_repertoire)
  A dynamically linked plugin function collation_tailoring() was added
  for the purpose of these tests.

- Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test
  They display two-dimensional charts showing which collations
  are compatible on which repertoires.

- Adding a number of MTR tests in the form of the originally
  reported stript for various collations:

    SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...';
Yuchen Pei
MDEV-41366 Check prefix key match in partition unordered index scan

The idea of Case 2 in can_skip_merging_scans is that when key prefix
is fixed, and the infix that is also the "partition by range" column,
we can scan each partition in order. This relies on accurately
returning EOF when a partition has no (more) matching rows. It is
possible to have no more matching rows at the first index read of a
partition in a LAST_OR_PREV access, in which case we need to check the
prefix match to rule out false positives and return EOF correctly
Vladislav Vaintroub
MDEV-11111: fix more buildbot failures

- main.mysqld--help: the three --embedded-* options are listed.
- debian: libmariadb3.symbols gets mariadb_set_embedded_hooks.
- mysqltest: allow 256 --server-arg (the private server of an embedded
  run gets more than 64 with the options of the test and of mtr).
- embedded runs: skip the tests that need a TCP listener, which the
  private server does not have (pool of threads, ssl_verify_ip,
  ssl_7937); main.variables sees skip_networking ON there.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Vladislav Vaintroub
MDEV-40955  mysql_client_test needs a resolvable DNS on Linux.

The test test_proxy_header_connect_errors_reset() relies on 192.0.2.50
(test IP, per RFC 5737) failing reverse DNS lookup permanently, which is
what a real, working resolver reports for it. Linux's resolver isn't so
RFC-compliant, when it has no route to any nameserver at all: it reports
EAI_AGAIN (temporary) instead, which is deliberately excluded from
connect-error accounting to avoid blocking hosts during a DNS outage.
That silently defeats the max_connect_errors check this test exercises.

Fix by forcing the deterministic "permanent failure" outcome via the
existing getnameinfo_error_noname debug instrumentation, same as its
sibling tests. Debug-only, like those siblings, since the workaround
needs DBUG_EXECUTE_IF.

Assisted-By: Claude Sonnet 5 <[email protected]>
Yuchen Pei
MDEV-41322 Convert KEY_NOT_FOUND to EOF in "unordered" partition index scans during index_next[_same]/index_prev calls

ha_partition::handle_unordered_scan_next_partition is called in a
variety of accesses, including index_read, index_prev, and index_next.

ha_partition::handle_unordered_next and
ha_partition::handle_unordered_prev are called from index_next and
index_prev accesses. They check for signs of end of
scan (m_part_spec.start_part == NO_CURRENT_PART_ID) and exit early if
that is the case. HA_ERR_END_OF_FILE (EOF) is commonly paired with
assigning NO_CURRENT_PART_ID to m_part_spec.start_part whenever
possible, to signal the end of scan.

The error HA_ERR_KEY_NOT_FOUND means the requested key is not found.
It should not mean the end of scan, when for example
ha_partition::handle_unordered_scan_next_partition is called from
index_read, because a subsequent index_next[_same] / index_prev call
would then incorrectly return immediately from
ha_partition::handle_unordered_next /
ha_partition::handle_unordered_prev. This is why HA_ERR_KEY_NOT_FOUND
is retained and returned in
ha_partition::handle_unordered_scan_next_partition.

But if the call is from index_next[_same] / index_prev,
HA_ERR_KEY_NOT_FOUND should indeed mean end of scan. In this patch, we
ensure this is the case by converting HA_ERR_KEY_NOT_FOUND to EOF in
ha_partition::handle_unordered_next /
ha_partition::handle_unordered_prev.
Oleksandr Byelkin
MDEV-41175 Fix stack-buffer-overflow in backup_log_ddl()

backup_log_ddl() built its log record in a fixed-size stack buffer,
but add_name_to_buffer() can expand each identifier character to 5
bytes when re-encoding it into my_charset_filename, so a RENAME TABLE
with long, special-character names overflowed the buffer.

Fixed by building the record in a growable String, converting names
directly into it, and setting backup_log_error instead of writing a
partial record if an allocation fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Pekka Lampio
MDEV-39143 Merge the type_mysql_json plugin into the server

A PR reviewer suggested folding the read-only type_mysql_json data-type
plugin into the server instead of keeping it a separate, optionally
loaded plugin, since MDEV-39143 now makes it a prerequisite for
replicating JSON from a MySQL master rather than an upgrade-time
convenience.
GoldenEmperor1177
MDEV-34215 Fix DATE_FORMAT result length for negative TIME

DATE_FORMAT calculated its maximum result length solely from the format
string. Formatting a negative TIME also prepends a minus sign, so the
cursor protocol trusted an undersized result and truncated -01 to -0.

Account for the possible sign in format_length() for TIME_FORMAT() and
DATE_FORMAT() with a native TIME argument. Enable the existing cursor
protocol regression cases for binary, latin1, and utf32 result character
sets.
Alexander Barkov
MDEV-41246 "Illegal mix of collations" on the mysql.user view

In progress
Georgi (Joro) Kodinov
more doxygen formatting added.
Marko Mäkelä
fixup! ea087a30f0b81dbf674d7b1fa2ddb852117818ea

Improve the InnoDB_backup::log_track() performance
Oleksandr Byelkin
MDEV-41175 Fix stack-buffer-overflow in backup_log_ddl()

backup_log_ddl() built its log record in a fixed-size stack buffer,
but add_name_to_buffer() can expand each identifier character to 5
bytes when re-encoding it into my_charset_filename, so a RENAME TABLE
with long, special-character names overflowed the buffer.

Fixed by building the record in a growable String, converting names
directly into it, and setting backup_log_error instead of writing a
partial record if an allocation fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>