Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

In one of the practical cloud MariaDB setups, a server node accesses its
datadir over the network, but also has a fast local SSD storage for
temporary data. The content of such temporary storage is lost when the
server container is destroyed.

The commit uses this ephemeral fast local storage (SSD) as an extension of
the portion of InnoDB buffer pool (DRAM) that caches persistent data
pages. This cache is separated from the persistent storage of data files
and ib_logfile0 and ignored during backup.

The following system variables were introduced:

innodb_extended_buffer_pool_size - the size of external buffer pool
file, if it equals to 0, external buffer pool will not be used;

innodb_extended_buffer_pool_path - the directory in which the external
buffer pool file is created, the data directory is used if the variable
is not set.

If innodb_extended_buffer_pool_size is not equal to 0, external buffer
pool file will be created on startup of a normal server instance. It is
created as a temporary file, so that the operating system removes it
when the server closes it or exits, and no stale file is left behind
after a crash. For this purpose row_merge_file_create_mode() was
generalized into pfs_create_temp_file() and moved to fil0fil.cc, so that
both the merge sort files and the external buffer pool file are created
by the same function.

Only clean pages will be flushed to external buffer pool file. There is
no need to flush dirty pages, as such pages will become clean after
flushing, and then will be evicted when they reach the tail of LRU list.
Freed pages are not written to the external buffer pool file either,
they are just evicted.

The general idea of this commit is to flush clean pages to external
buffer pool file when they are evicted.

A page can be evicted either by transaction thread or by background
thread of page cleaner. In some cases transaction thread is waiting for
page cleaner thread to finish its job. We can't do flushing in external
buffer pool file when transaction threads are waiting for eviction,
that would hurt performance. That's why the only case for flushing is
when page cleaner thread evicts pages in background and there are no
waiters. For this purpose buf_pool_t::done_flush_list_waiters_count
variable was introduced, we flush evicted clean pages only if the
variable is zeroed.

Clean pages are evicted in buf_flush_LRU_list_batch() to keep some
amount of pages in buffer pool's free list. That's why we flush every
second page to external buffer pool file, otherwise there could be not
enough amount of pages in free list to let transaction threads to
allocate buffer pool pages without page cleaner waiting. This might be
not a good solution, but this is enough for prototyping.

External buffer pool page is introduced to store information in buffer
pool page hash about the certain page can be read from external buffer
pool file. The first several members of such page must be the same as the
members of internal page. External page frame must be equal to the
certain value to distinguish external page from internal one. External
buffer pages are preallocated on startup in external pages array. We
could get rid of the frame in external page, and check if the page's
address belongs to the array to distinguish external and internal pages.

There are also external pages free and LRU lists. When some internal page
is decided to be flushed in external buffer pool file, a new external
page is allocated either from the head of external free list, or from
the tail of external LRU list. Both lists are protected with
buf_pool.mutex. It makes sense, because a page is removed from internal
LRU list during eviction under buf_pool.mutex.

Then internal page is locked and the allocated external page is attached
to io request for external buffer pool file, and when write request is
completed, the internal page is replaced with external one in page hash,
external page is pushed to the head of external LRU list and internal
page is unlocked. After internal page was removed from external free list,
it was not placed in external LRU, and placed there only
after write completion, so the page can't be used by the other threads
until write is completed.

Page hash chain get element function has additional template parameter,
which notifies the function if external pages must be ignored or not. We
don't ignore external pages in page hash in two cases, when some page is
initialized for read and when one is reinitialized for new page creating.

When an internal page is initialized for read and external page with the
same page id is found in page hash, the internal page is locked,
the external page in replaced with newly initialized internal page in the
page hash chain, the external page is removed from external LRU list and
attached to io request to external buffer pool file. When the io request
is completed, external page is returned to external free list,
internal page is unlocked. So during read external page is absent in both
external LRU and free lists and can't be reused.

When an internal page is initialized for new page creating and external
pages with the same page id is found in page hash, we just remove external
page from the page hash chain and external LRU list and push it to the
head of external free list. So the external page can be used for future
flushing.

The external buffer pool file is not represented by a fil_space_t. The
requests to it are issued by fil_system_t::ext_bp_io(), and the external
buffer pool page which a request refers to is stored in IORequest in
place of the fil_node_t. The pages are written to and read from the
external buffer pool file in the same form in which they are written to
their tablespaces, i.e. compressed and encrypted pages stay compressed
and encrypted in external buffer pool file.

If a write to the external buffer pool file fails, a warning is written
to the error log and the external buffer pool is disabled for the rest
of the server lifetime.

Currently the commit passed some local smoke tests, mtr and RQG tests
with external buffer pool turned on.

TODO:
1. Add some monitoring, i.e. how much pages are currently in external
  buffer pool, the percent of hits during reading, take a look at the
  current buffer pool monitoring and implement the general monitoring
  tools for external buffer pool.
2. Think about partial initialization of external pages array, as it was
  done for internal pages.
3. Take a look at compressed LRU list, it looks like currently it's not
  covered with eviction to external buffer pool file (I don't currently
  understand if we need it at all).
4. Think about more suitable algorithm for eviction to external buffer
  pool, currently just every second page is flushed.
Yuchen Pei
MDEV-25848 Allow "js IS NOT NULL AND <JSON_SARGABLE_PREDICATE on js>" to use range access
Georgi (Joro) Kodinov
MDEV-42338: Treat openssl,pcre and zlib library headers as "system"

Mark the 3d party library headers as "system" headers so that the
warnings in these files are suppressed for supporting compilers.
Andrei Elkin
MDEV-42337 no semi-sync kill of a closed connection

This is MDEV-42004 fixes follow-up.
Repl_semi_sync_slave::kill_connection() reconnected to the master at the IO thread exit even though end_server() had already closed the connection (e.g. STOP SLAVE interrupted a reconnect), so the stop waited rpl_semi_sync_slave_kill_conn_timeout once more.
Return early when mysql->net.vio is NULL.
Andrei Elkin
MDEV-42004 STOP SLAVE waits for slave_net_timeout when master is unreachable

STOP SLAVE could take up to slave_net_timeout to stop the slave IO thread
when the master accepts the connection but does not answer, e.g. when its
host is frozen or its packets are dropped.
The IO thread was blocked in a reconnect
  try_to_reconnect() -> safe_reconnect() -> connect_to_master() ->
  mysql_reconnect() -> mysql_real_connect()
waiting for the greeting of the master, and nothing could wake it up in the
base of this patch, 11.4 at c5272f132d3. The stall is not specific to
semi-sync: an asynchronous slave waits the same, and a semi-sync one waits
in addition for rpl_semi_sync_slave_kill_conn_timeout at the IO thread exit,
which this commit does not change.

The wait was interruptible before MDEV-32567 removed the pthread_kill() of
the slave thread in terminate_slave_thread() (commit 013fc02a23e) in order to
rely on the THD::awake_no_mutex() -> vio_shutdown(active_vio) mechanism.
This bug exposes that it does not cover slave reconnect, as the slave's IO
thread does not bind its THD::active_vio to a new socket vio at the time of
the reconnect attempt.

This commit makes that "tentative" vio reachable for awake_no_mutex().
For that mysql_real_connect() now publishes the vio to the slave thread
right after it is created.
See the new hook slave_io_thread_attach_vio() in the TCP branch (the Unix one
is irrelevant for slave) that pairs an existing slave_io_thread_detach_vio().
So now in the patch when STOP SLAVE shuts the vio down, which wakes up
the poll().
The detach hook is called when the bound to THD::active_vio is about
to be destroyed.

connect_to_master() reports the interrupt to the error log when
log_warnings > 2 and the slave has been stopped after the failed connect:

  Slave I/O thread: reconnect to master '[email protected]:19000' was
  interrupted by the slave stop: Lost connection to server at 'reading
  initial communication packet', system error: 0 ...

Tested with rpl.rpl_stop_slave_unreachable_master.

The test simulates an unresponsive master: it is not shut down but frozen
with SIGSTOP, so the kernel still completes the TCP handshake of a new
connection, while no greeting is sent, and the IO thread gets into its
reconnect. The test waits for that to most probably happen and then
runs STOP SLAVE, which normally (not on an overloaded server) must not wait
for slave_net_timeout.
A shut-down master would not serve for that: its port is closed, so the
connect fails at once with "Connection refused", and the IO thread, which
then sleeps connect_retry in slave_sleep() (it checks io_slave_killed()),
stops promptly without this fix.
The test is not run on Windows, as it needs SIGSTOP.

With --let $debug_test= 1 the test prints how long STOP SLAVE took. Measured
this way, it returned after 3.5 s before the fix and 1.0 s after, with
slave_net_timeout=3 and rpl_semi_sync_slave_kill_conn_timeout=1; the same
test with a plain asynchronous slave took 2.5 s and 0.0 s.
For a reliable run on BuildBot the time is not checked. The fix is proven
by the error text in the note above. A reconnect that waits for the greeting
until the timeout is out fails with "waiting for initial communication
packet ... Connection timed out", while the one whose vio is shut down fails
with "reading initial communication packet". The test greps the slave's
error log for the latter with search_pattern_in_file.inc, which reports

  FOUND 1 /Slave I/O thread: reconnect to master .* was interrupted by the slave stop: .*reading initial communication packet/ in mysqld.2.err
Sergei Golubchik
MDEV-42324 CONNECTION of a MULTIPLE table is used as a format string

For ODBC tables with MULTIPLE=1 TDBODBC::GetFile() replaced the DBQ
value in CONNECTION with "%s" and TDBODBC::SetFile() passed the result
to snprintf() as the format string.

Store the parts of CONNECTION before and after the file name as two
strings and concatenate them with the file name.

Assisted-by: Claude:claude-opus-5-5
Kristian Nielsen
Include binlog in backup when --binlog-storage-engine=innodb

Collect the set of binlog files to include in the backup at backup commit
time, consistent with the backup LSN. Then copy the binlog files during the
FINISH stage, where locks are released; this is safe because binlog files
are append-only and recovery clears any extra data written after the backup
LSN.

This can be used to provision a slave from a backup of a master, taking the
starting GTID position from the @@gtid_binlog_pos. An mtr test case
binlog_in_engine.server_backup_slave_provision demonstrates this use case.

Signed-off-by: Kristian Nielsen <[email protected]>
Marko Mäkelä
MDEV-39091 BACKUP SERVER of ENGINE=RocksDB

RocksDB is based on log structured merge trees (LSM trees)
that are immutable files. A consistent point-in-time snapshot
is obtained at BACKUP_LOCK_COMMIT cheaply via rdb_create_checkpoint(),
which is also what mariadb-backup is invoking via
SET GLOBAL rocksdb_create_checkpoint.

Under BACKUP_LOCK_COMMIT, a backup snapshot is created in #rocksdb under
rocksdb_datadir (by default, #rocksdb/#rocksdb in the server datadir).
This snapshot will be made available in #rocksdb subdirectory of the
backup target. The default rocksdb_datadir="./#rocksdb" makes restore
transparent: just point datadir to the backup directory or a restored
copy of it.

The transfer of the snapshot is deferred to BACKUP_FINISHED, after the
backup lock has been released, and finally the checkpoint is removed
from rocksdb_datadir.

Co-developed by: Thirunarayanan Balathandayuthapani
Yuchen Pei
MDEV-25848 bound range optimizer work for many ARRAY index points

With one range slot per ARRAY index, the number of points in a
SEL_ARG_ARRAY is no longer bounded by MAX_KEY, and a JSON_OVERLAPS or
JSON_CONTAINS constant with N elements cost (see the TODOs):

* N records_in_range() index dives, one per leaf, even when the
  estimate is never used;
* O(N^2) work in key_or_array(), which copied both point lists into a
  new SEL_ARG_ARRAY on every OR and deduplicated with a linear scan.

Now:

* key_or_array() appends the smaller point list to the larger
  SEL_ARG_ARRAY in place, or to a copy of it if it is shared
  (use_count > 0), without sorting or deduplicating.
* SEL_ARG_ARRAY::normalize() sorts and deduplicates the points once,
  when they are needed: estimates, quick select, optimizer trace.
* SEL_ARG_ARRAY::estimate() computes rows and cost lazily, on first
  use (key_and() or check_quick_select_array()). With
  eq_range_index_dive_limit or more points it dives on that many
  points spread over the sorted list and extrapolates.
* JSON_CONTAINS skips duplicate elements of the constant and uses at
  most eq_range_index_dive_limit distinct ones for the range, since
  each AND costs a dive and fewer conjuncts give a superset.

Points are now read in sorted order, so the order of rows from an
ARRAY index range scan may differ.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

MDEV-25848 deduplicate JSON_OVERLAPS/JSON_CONTAINS elements when parsing

Skip duplicate elements of the constant in json_get_range_elements(),
using a hash on the element's JSON text, instead of in
json_get_func_mm_tree() for JSON_CONTAINS only. Duplicates now never
become Items, SEL_ARG_ARRAY leaves or make_key() calls, for
JSON_OVERLAPS as well, and json_get_func_mm_tree() only counts the
distinct elements against eq_range_index_dive_limit.

The text includes the quotes of strings, so "1" and 1 are different.
Equal values with different text, such as 1 and 1.0, are still
deduplicated later by SEL_ARG_ARRAY::normalize().

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Aleksey Midenkov
MDEV-41645 Auto-create of SYSTEM_TIME history partitions checks the partition limit without existing partitions and subpartitions

partition_info::vers_set_hist_part() counts the history partitions to
auto-create for INTERVAL ... AUTO and stopped when the count reached
MAX_PARTITIONS - 2. The limit compared only the number of new partitions
with MAX_PARTITIONS - 2. It ignored the partitions the table already has
and the number of subpartitions, so the real total
((num_parts + create_count) * num_subparts) could be well over
MAX_PARTITIONS while this check passed. The early error was then not
given, and the limit was left to whatever checks the actual partition
creation does.

The fix compares the total of the existing and the new partitions, times
the number of subpartitions, with MAX_PARTITIONS.
Marko Mäkelä
Add my_dir_open() written by Monty
Georgi (Joro) Kodinov
MDEV-42338: Treat openssl,pcre and zlib library headers as "system"

Mark the 3d party library headers as "system" headers so that the
warnings in these files are suppressed for supporting compilers.
Marko Mäkelä
MDEV-39091 BACKUP SERVER of ENGINE=RocksDB

RocksDB is based on log structured merge trees (LSM trees)
that are immutable files. A consistent point-in-time snapshot
is obtained at BACKUP_LOCK_COMMIT cheaply via rdb_create_checkpoint(),
which is also what mariadb-backup is invoking via
SET GLOBAL rocksdb_create_checkpoint.

Under BACKUP_LOCK_COMMIT, a backup snapshot is created in #rocksdb under
rocksdb_datadir (by default, #rocksdb/#rocksdb in the server datadir).
This snapshot will be made available in #rocksdb subdirectory of the
backup target. The default rocksdb_datadir="./#rocksdb" makes restore
transparent: just point datadir to the backup directory or a restored
copy of it.

The transfer of the snapshot is deferred to BACKUP_FINISHED, after the
backup lock has been released, and finally the checkpoint is removed
from rocksdb_datadir.

Co-developed by: Thirunarayanan Balathandayuthapani
Reviewed by: Alessandro Vetere
kjarir
MDEV-10526: Add binary string support to bitwise operators and aggregates

Previously all bitwise operators and BIT_AND/BIT_OR/BIT_XOR converted
their arguments to BIGINT, silently truncating values wider than 64 bits.
This made them useless for INET6 addresses, UUIDs and other binary data.

When the arguments are binary strings (BINARY, VARBINARY, BLOB, but not
hex hybrids like 0xFF), the operators &, |, ^, ~, << and >> and the
aggregate functions BIT_AND, BIT_OR and BIT_XOR now work byte by byte
and return a binary string of the length of the operand(s).

Scalar operators:
- Implemented as Handler_str subclasses of the existing Item_handled_func
  handlers. Func_handler_bit_bin_to_bin and
  Func_handler_bit_op2_bin_to_bin hold the common code.
- Operands of different lengths give ER_INVALID_BITWISE_OPERANDS_SIZE
  as a warning and the result is NULL.
- The result collation is aggregated from the binary string argument(s),
  as for other string functions.

Aggregate functions:
- A binary accumulator initialized to the neutral value (0xFF for
  BIT_AND, 0x00 for BIT_OR and BIT_XOR).
- The result length is the length of the aggregated values, which may
  be shorter than the declared length (e.g. INET6_ATON() of an IPv4
  address). All values of a group must have the same length, otherwise
  ER_INVALID_BITWISE_OPERANDS_SIZE is raised.
- An empty set returns the neutral value of the declared length.
- The operand is limited to 512 bytes,
  ER_INVALID_BITWISE_AGGREGATE_OPERANDS_SIZE otherwise.
- The result data type is VARBINARY. val_int(), val_real(),
  val_decimal() and get_date() convert the binary string the same way
  as for the scalar operators.
- Window functions use per-bit counters, as for the 64-bit mode.
- reset_field()/update_field() support the binary mode for GROUP BY
  with a temporary table. Rows rejected by a FILTER clause (MDEV-24943)
  leave the field unchanged.
- direct_add() for a binary partial result, used by Spider.

MDEV-10526: Add Spider pushdown for bitwise aggregates

Push BIT_AND, BIT_OR and BIT_XOR down to the data nodes and merge the
partial results locally with direct_add(), in both integer and binary
string mode. The functions are associative and commutative, so the
partial results can be combined like for SUM, MIN and MAX.

In integer mode the partial result is parsed with strntoull10rnd():
row->val_int() overflows for values above LONGLONG_MAX.
A partial result that can not be parsed is an error, as any substitute
value would silently corrupt the merged result.

Aggregates with a FILTER clause are not pushed down.

MDEV-10526: Address review: test coverage, Spider empty data nodes

Add tests for the code paths reported as not covered:
- scalar operators: NULL in the first operand, empty strings, shift
  counts inside a byte, across bytes, out of range and negative
- aggregates: the binary result used as an integer, a boolean and a
  date, the integer result as a string, a double, a decimal and a date,
  data types that are not allowed, values of different lengths in a
  temporary table group and in a window, sliding frames removing
  values (also with FILTER), window functions over WITH ROLLUP
- Spider: groups whose partial results come from several data nodes
  (update_field() with direct_add()), empty data nodes, partial
  results that are not an integer, too long or NULL
- out of memory when allocating the result of an operator and the
  bit counters of a window function (debug builds)

Fix the bugs found by the new tests:
- A binary aggregate pushed down to Spider failed with
  ER_INVALID_BITWISE_OPERANDS_SIZE if the first data node had no rows
  and the values were shorter than the declared length. The neutral
  value of the declared length returned by such a node was taken as
  the first value. A neutral partial result is now always skipped,
  and ordinary values are not checked for it any more, so values of
  different lengths are always an error.
- A partial result longer than the declared length of the argument
  (the data node has a wider column) was written past the end of the
  accumulator. It is now rejected.

Remove the code that could not be reached:
- BIT_AND, BIT_OR and BIT_XOR never return NULL, so Item_sum_bit does
  not need to handle a NULL partial result or a NULL temporary table
  field. Spider returns an error for a NULL.
- The binary accumulator and the partial result are now kept in
  buffers inside Item_sum_bit: they are at most 512 bytes long. There
  is no allocation that can fail any more.
- The operators allocate the result in one common method, which also
  handles empty strings.
- Window functions never aggregate values of different lengths, and
  are never copied by copy_or_same(), this is now asserted.

MDEV-10526: Fix Spider bitwise aggregate tests for --ps-protocol

The prepared statement protocol changes the value of the
Spider_direct_aggregate status variable for the GROUP BY query, and
the stored procedure protocol adds a note to SHOW WARNINGS after an
error. Run that query without the prepared statement protocol and
show only the warnings of the query.
Alexey Botchkov
Excessive json_nice() calls eliminated.

Item_json_func descendants now have val_json() doing
the job, returning the valid JSON as it is.
The val_str() gets the val_json() return and trims it with json_nice().
Functions that surely return the nice json just return the val_json()
in the val_str().
Thirunarayanan Balathandayuthapani
MDEV-39061 mariadb-backup compatible wrapper for BACKUP SERVER

This adds a shell script that lets users keep using their existing
mariadb-backup commands while the real work is done by the new
server-side BACKUP SERVER command. The goal is "drop-in": users
should not have to change their backup scripts.

mariadb-backup-server.sh : Understands the usual mariadb-backup
modes and translates each one.

mbstream-server.sh : lets streamed backups be unpacked by
pipelines that expect the mbstream CLI

mariadb-backup-pipe.sh is the stream sink that
BACKUP SERVER itself runs.

All three are documented in README.md.

--backup
========
Connects with the mariadb client and runs
"BACKUP SERVER TO '<dir>'".

Connection options (--user, --host, --port, --socket,
--defaults-file, ssl, ...) are passed through to the client.

--parallel=N becomes "<N+1> CONCURRENT": mariadb-backup
runs a dedicated log_copying_thread() besides its N data-copy
threads, while CONCURRENT counts every BACKUP SERVER worker.
The result is clamped to the 1..256 the parser accepts,
and to a minimum of 2.

After the backup it writes backup-prepare.cnf into the backup
directory, recording what --prepare needs later:
  - where mariadbd lives
  - InnoDB parameters (page size, data file path, undo tablespaces,
                      checksum algorithm, log file size)
  - If the server is encrypted then how to reload the
    encryption key plugin (the file_key_management variables),
    so an encrypted backup can be prepared without extra input.
  - every active provider_* compression plugin, with plugin-dir.

The providers matter because --prepare runs "mariadbd --bootstrap
--defaults-file=backup-prepare.cnf", which suppresses the
/etc/my.cnf.d snippet that normally enables a provider. Recovery has
to decompress a PAGE_COMPRESSED page before it can apply redo to it,
so without the provider it aborts with "Unable to apply log to
corrupted page". mariadb-backup carries the same plugins through
backup-my.cnf (encryption_plugin.cc QUERY_PLUGIN collects the
ENCRYPTION plugin and the provider_* DAEMON plugins together), so
this is parity with it rather than a new capability.

--backup --stream
=================
Runs "BACKUP SERVER WITH 'pipe'".
The server only accepts a bare command name, which it resolves
n the PATH of the mariadbd process after prepending mariadb-backup-,
so uses the installed mariadb-backup-pipe instead.

mariadb-backup-pipe writes the tar into
.mariadb-backup-pipe-<stream>.tar, a named pipe the wrapper
created in the server @@datadir and is already draining to
its own stdout.

The backup therefore reaches the consumer as it is produced and
never lands on local disk. The wrapper appends backup-prepare.cnf
as a final tar afterwards; the server's tar carries no
end-of-archive marker, so that trailing archive supplies the
only one and the whole stream extracts with a plain "tar -x".
The trailing archive is written with "tar -b 1" so that nothing
follows its end-of-archive marker. At the default blocking factor tar
pads the archive to 10240 bytes; the reader stops at the marker and
exits with the padding still unwritten, which on a small pipe is a
short write. tar treats a short write as fatal, so a backup that had
in fact completed was reported as failed.

--parallel is ignored here, with a warning.

Four properties follow from how BACKUP SERVER streams,
all differing from mariadb-backup:
- local: the stream command runs inside the server,
so the wrapper must share its filesystem;
- fifo lives in @@datadir, because mariadb-backup-pipe resolves it
relative to the working directory of mariadbd, so --stream needs
write access there;
- tar only: any --stream=<format> yields tar;
- single-threaded: one worker, so no parallel read either.
--target-dir is optional in stream mode;

mbstream-server.sh maps the mbstream CLI onto a plain
"tar -x"/"tar -c", so existing "mbstream -x"/"-c" pipelines
keep working on the wrapper's stream.

mbstream-only flags (-p/--parallel, ...) are accepted and
ignored; any other unknown option is rejected.

Environment overrides:
MARIADB (client),
MARIADBD (the --prepare bootstrap server) and
TAR (the tar implementation, e.g. TAR=bsdtar) can each be overridden.

To run the bootstrap under rr, put it in MARIADBD and
let rr's own _RR_TRACE_DIR choose the trace location, e.g.
  _RR_TRACE_DIR=/dev/shm/rr MARIADBD='rr record mariadbd'

--prepare
=========
Starts "mariadbd --bootstrap" on the backup directory using
backup-prepare.cnf as its defaults file, replays the archived redo
log between the start and target LSN read from backup.cnf,
then builds a fresh ib_logfile0 so a normal server can start
on the directory.

mariadbd is taken from the path recorded in backup-prepare.cnf
if that binary exists, otherwise by searching
/libexec, /sbin, /bin and the configured install directories.

PATH is not searched; set MARIADBD to point elsewhere.
User --defaults-file/-extra-file and encryption options are
layered onto the bootstrap.

--copy-back / --move-back
=========================
Copy or move a prepared backup into the datadir. The datadir
is created if missing, a non-empty datadir is refused unless
--force-non-empty-directories is given, and a chown
reminder is printed.

If --aria-log-dir-path is given, the Aria logs (aria_log_control,
aria_log.*) are relocated into that directory.

Packaging
=========
The wrapper is not installed by default and never replaces the
real mariadb-backup / mbstream binaries.
1. cmake -DWITH_MARIABACKUP_WRAPPER=ON (default OFF) controls it.
2. When ON, the scripts install as /usr/bin/mariadb-backup-server,
/usr/bin/mbstream-server and /usr/bin/mariadb-backup-pipe, tagged
COMPONENT Backup so they ship in the mariadb-backup package.
mariadb-backup-pipe must end up in the PATH of the mariadbd
process, not merely in the PATH of whoever runs the wrapper.
  mariadb-backup-pipe is generated unconditionally, because the
  mariabackup test suite needs it on the server's PATH even when the
  wrapper is not installed; only its install rule is gated.
3. RPM: nothing extra to do. the component handles it.
4. DEB: not wired. debian/rules uses --fail-missing and does not
enable the option, so the -server binaries are not listed.
To ship via DEB, make a paired change: add
-DWITH_MARIABACKUP_WRAPPER=ON in debian/rules and list all three of
usr/bin/mariadb-backup-server, usr/bin/mbstream-server and
usr/bin/mariadb-backup-pipe in debian/mariadb-backup.install together.
5. The real mariadb-backup/mbstream binaries and the
mariabackup symlink are left untouched; opt in via an alias or a
symlink early in PATH.

Limitations (not supported yet)
===============================
1) Incremental backup & prepare (--incremental-basedir,
  --incremental-dir, --apply-log-only)
2) --rollback-xa
3) Partial backup (--databases, --tables, --tables-file)
4) Output compression and encryption (--compress, --encrypt)
5) --export is accepted but only warns and runs a plain recovery
6) --extra-lsndir is ignored
7) --parallel is ignored with --stream
8) Windows: POSIX sh only, not installed on Windows

Behaviour differences from native mariadb-backup
================================================
- The wrapper needs the mariadb client on PATH for --backup;
--prepare needs mariadbd recorded in backup-prepare.cnf, in a
standard install directory, or named by MARIADBD
- BACKUP SERVER refuses an already-existing target directory
- BACKUP SERVER does copy the data file as raw pages without
checksum validation, so a corrupted table is not detected
at backup time
- --prepare only works on a wrapper-made backup: it
needs backup-prepare.cnf
- --stream is tar, not xbstream, local-only and single-threaded,
and needs write access to @@datadir for the fifo

Tests
=====
include/have_mariabackup_wrapper.inc redirects $XTRABACKUP to
mariadb-backup-server.sh and $XBSTREAM to mbstream-server.sh,
skipping when a wrapper or the mariadb client is unavailable.

include/have_mariabackup_combination.inc runs a test under both the
[CLIENT] mariadb-backup binary and the [SERVER] wrapper.

suite/mariabackup/suite.pm prepends the directory holding
mariadb-backup-pipe to PATH. BACKUP SERVER resolves the command in the
PATH of the mariadbd process, which mtr starts from the suite's
environment, so it has to be set there rather than with a --let at
--exec time. have_mariabackup_wrapper.inc skips the [SERVER]
combination when the helper is not on PATH.
Andrei Elkin
MDEV-42337 no semi-sync kill of a closed connection

This is MDEV-42004 fixes follow-up.
Repl_semi_sync_slave::kill_connection() reconnected to the master at
the IO thread exit even though end_server() had already closed the
connection (e.g. STOP SLAVE interrupted a reconnect), so the stop
waited rpl_semi_sync_slave_kill_conn_timeout once more.  Return early
when mysql->net.vio is NULL.
Kristian Nielsen
After-review fixes

Signed-off-by: Kristian Nielsen <[email protected]>
Kristian Nielsen
Add testcase for streamed backup with binlog-in-engine

Signed-off-by: Kristian Nielsen <[email protected]>
Yuchen Pei
MDEV-25848 one range slot per ARRAY index

Before, every MEMBER OF / JSON_OVERLAPS / JSON_CONTAINS point got its
own cloned KEY_PART and key slot, and points were combined only through
index_merge (OR) or by picking one slot (AND).

Now all points on an ARRAY key part share the original slot:

* SEL_ARG_ARRAY holds a list of make_key() points ("the array contains
  any of them") and the summed records_in_range() estimate and cost,
  taken once when the point is built.
* key_or() of two ARRAY ranges takes the union of the points;
  key_and() keeps the operand with the lower estimate, the WHERE
  condition checks the rest.
* KEY_PART cloning (get_claimed_key_part()) is now RTREE-only, so
  ARRAY-only queries size SEL_TREE::keys by param->keys again.
* QUICK_RANGE_SELECT_ARRAY gets one QUICK_RANGE per point and passes
  all points to hlindex_read_first(); json_index reads them in turn
  and skips trefs it has already returned.
* check_quick_select_array() uses the stored estimate, and rejects
  non-ARRAY SEL_ARGs (e.g. IS NULL) on the ARRAY key part.

ORed MEMBER OF and JSON_OVERLAPS now give a single range scan instead
of index_merge sort_union, with no MAX_KEY limit on the number of
points.
Marko Mäkelä
MDEV-14992/MDEV-38362/MDEV-39092 BACKUP SERVER

The following SQL statements will be introduced:

BACKUP SERVER TO '/path/to/directory' [ 1 CONCURRENT ];
BACKUP SERVER WITH [ 1 CONCURRENT ] 'command';

In place of the 1, any positive number of threads may be specified.
For the first variant, '/path/to' must exist and '/path/to/directory'
must not exist; that is where the backup will be written to.

For BACKUP SERVER WITH, the string mariadb-backup-
will be prepended to the specified command and quoted.
Path separators or quotes are not allowed in command.
We expect the mariadb-backup-command to be in the
search PATH of the mariadbd process.
For now, we distribute no such script or program;
the user has to write one, something like this:

zstdmt|exec ssh [email protected] "exec cat > $1.tar.zstd"

The standard input of that command will be in a format compatible with
GNU tar --format=oldgnu (and also BSD tar variants that are also part of
Microsoft Windows and Apple macOS). The command is expected to optionally
compress and encrypt the stream and redirect it to a file on a local or
a remote server. The BACKUP SERVER WITH will append an additional argument,
a positive base-ten number in ASCII, starting with 1, to identify the
current thread. In this way, there can be multiple concurrent streams.
The InnoDB write-ahead log will be streamed near the end, because
the stream format requires the file sizes to be declared upfront.

In BACKUP SERVER TO ... 2 CONCURRENT or more, one thread will be
responsible for copying the InnoDB write-ahead log from the beginning
of the backup, running InnoDB_backup::log_track(). Other files will be
copied by other threads.

Note: In single-threaded BACKUP SERVER TO, the parameter
innodb_log_recovery_start that is written out to backup.cnf is
STRICTLY NECESSARY TO AVOID CORRUPTION during recovery! By default,
InnoDB crash recovery starts from the latest available log
checkpoint. However, for restoring a backup, recovery must start from
the checkpoint that was the latest when the backup was
started. Starting recovery from a possible later checkpoint will
result in a corrupted database!

The backup or the first stream will contain a file backup.cnf, which
includes parameters needed for restoring the backup. Currently,
these are innodb_log_recovery_start and innodb_log_recovery_target.
If innodb_log_recovery_target>0, InnoDB will be in read-only mode,
not allowing any writes to persistent files other than via the log
application.

To restore a streaming backup made with BACKUP SERVER WITH, an empty
directory needs to be created and all streams be extracted there using
the standard tar utility of the operating system, optionally after
undoing any encryption or compression that had been added by the
backup command. Then, the backup is prepared or MariaDB server started
up on the extracted directory, similar to as if the BACKUP SERVER TO
statement had been used.

The following will be implemented separately:

MDEV-39061 mariadb-backup compatible wrapper script for BACKUP SERVER
MDEV-40163 Partial backup and restore
MDEV-39091 Back up ENGINE=RocksDB
MDEV-40333 Concurrent DDL for Aria tables in BACKUP SERVER
MDEV-39987 Improve BACKUP SERVER performance for ENGINE=Aria

The implementation introduces a basic multi-threaded driver
Sql_cmd_backup, storage engine interfaces, and basic copying of the
storage engines InnoDB, Aria, MyISAM, MERGE (MyISAM), Archive, CSV.

backup_target: A target directory and a set of operations,
such as backup_target::create_file().
On Microsoft Windows, we must use directory paths because there is
no variant of CopyFileEx() that would work on file handles.

backup_sink: Per-thread storage engine context and output stream,
and a set of operations such as backup_sink::create_file() that
are valid if an output stream exists. The member "id" identifies
the thread, between 0 and CONCURRENT-1.

backup_context: Process-wide BACKUP SERVER context.
Handles the directory traversal and copying of
files for built-in storage engines that do not
implement this backup interface.

backup_target_stage: Wrap backup_context.

backup_target_stage::step(),
backup_context::step(): Process a file
from a directory scan, or by invoking
handlerton::backup_step().

handlerton::backup_file(): Check if a file should be
included in the backup. Implemented for ENGINE=Aria
in maria_backup_file().

handlerton::backup_start(), handlerton::backup_end(): Invoked at the
start or end of a backup phase, in the thread that executes a
BACKUP SERVER statement.

handlerton::backup_step(): A backup step that can be invoked from
multiple threads concurrently, between the execution of the corresponding
handlerton::backup_start() and handlerton::backup_end() of the same
phase.

my_clone_file_range(): A wrapper for Linux block-cloning ioctl(2)
FICLONERANGE, which may fail depending on the parameters
and thus require a fallback to my_copy_file_range().

my_copy_file_range(): A partial or sparse file-copying service.
On FreeBSD, this simply wraps copy_file_range(2).
On Linux, for small files we attempt copy_file_range(2) before
falling back to specialised a pread/pwrite loop, which turned out
to be the fastest way to copy large files. Even though sendfile(2)
would work for copying between files, it as well as
copy_file_range(2) can be much slower than the alternatives.
For large files on Linux, we always invoke a pread/pwrite loop
with special cache control. It is also needed as a fallback,
because the Linux copy_file_range(2) can fail depending on the
file system.

On Windows, my_copy_file_range() is not usable on files that have
been opened for asynchronous I/O, such as InnoDB data files.
Such files have to be copied via backup::copy(), which uses the
native_file_handle wrapper of HANDLE.

my_copy_file(): Copy files between two POSIX file descriptors.
On Microsoft Windows, my_copy() or CopyFileEx() should be used instead.
On Linux, we attempt the ioctl(2) FICLONE before falling back to
my_copy_file_range().

my_copy(): Make use of CopyFileEx() on Microsoft Windows.

backup::copy(): Wraps my_copy_file_range() or the
Windows native_file_handle based copied outside mysys.

backup::copy_or_stream(): Copy or stream a file.

backup_sink::append(): The streaming streaming equivalent of backup::copy().

backup_sink::append_async(): A variant of backup_sink::append()
where the source file region is guaranteed to be immutable after the
call returns. Zero-copy Linux sendfile(2) is inherently risky for copying
data files that may be modified in place, because it could introduce
a race condition between a page write that runs concurrently with a
child process that is reading the data from the pipe. This could be
improved with io_uring later.

backup_sink::zeropad(): Zero-pad the last tar block if needed.

backup_sink::write(): Write to a stream or a file, with a retry
mechanism in case a partial write occurred.

pread_write(): Allocate a buffer of up to 4 MiB.
This is the "slow path" of copying or streaming files.
For InnoDB data files, BACKUP SERVER TO invokes this on output that
has been opened in unbuffered I/O (O_DIRECT) mode, to avoid polluting
the Linux file system cache (block cache). For the InnoDB log,
regular buffered I/O performed best in our tests.

maria_backup_start(): The entry point for ENGINE=Aria.
Currently, the directory traversal is single-threaded and a memory hog.
This is to be improved in MDEV-39987. The Aria log and data file
copying were implemented by Michael 'Monty' Widenius and
Alessandro Vetere.

maria_backup_log_tail(): Copy the necessary part of the last
ENGINE=Aria log, by my_clone_file_range() or my_copy_file_range().
The copy will be trimmed to the logical end of the backup.
The logic was developed by Alessandro Vetere.

InnoDB_backup::init(): Wait for a possible previous BACKUP SERVER
operation to reach the very end of InnoDB_backup::context::cleanup()
so that the context can be safely reused.

InnoDB_backup::queue: Collection of tablespace IDs and payload sizes
at the start of the backup, and the log_sys.first_lsn of log files
that have to be included in the backup. If any data file is created or
extended while the backup is executing, we must have the corresponding
write-ahead-log entries that we are copying since the latest
checkpoint that was completed when the backup started. If any
tablespaces are deleted during the backup, we may or may not copy
them, and the application of a FILE_DELETE record will remove them.
Similarly, applying FILE_RENAME or FILE_CREATE records will rename or
create files during recovery as needed.

InnoDB_backup::context::tracked: Log file queue for the dedicated
log-copying task InnoDB_backup::log_track().

innodb_backup_log_tracking(): Determine if log-tracking backup is
active, which blocks concurrent changes of innodb_log_file_buffering
and innodb_log_file_write_through.

log_sys.backup: Whether BACKUP SERVER is in progress. The purpose of this
is to make BACKUP SERVER prevent the concurrent execution of
SET GLOBAL innodb_log_archive=OFF or SET GLOBAL innodb_log_file_size
when innodb_log_archive=OFF.

log_sys.archived_checkpoint: Keep track of the earliest available
checkpoint, corresponding to log_sys.archived_lsn. This reflects
SET GLOBAL innodb_log_recovery_start (which is settable now), for
incremental backup.

fil_system.have_all_spaces: Whether all tablespace metadata is guaranteed
to be known. To speed up startup, InnoDB does not normally open
all tablespace files.

fil_space_t::create_lsn: Change to Atomic_relaxed and use this to
indicate tablespace creation LSN, in addition to indicate undo
tablespace rebuild LSN.

fil_space_t::backup_end: The first page number that is not being backed up
(by default 0, to indicate that no backup is in progress).

fil_space_t::BACKUP_BATCH_SIZE: The number of preceding pages that will be
covered by fil_space_t::backup_end. This is the unit of "page range locking"
during InnoDB backup.

fil_node_t::backup_name: A snapshot of the file name, sampled in
InnoDB_backup::init(). For backing up any files that were created before
the backup start checkpoint LSN, these names will be written to the backup.
On restore, the files will be renamed or deleted as needed, by
applying FILE_RENAME or FILE_DELETE records.

fil_node_t::rename(), fil_node_t::set_backup_name(),
fil_node_t::get_backup_name(), fil_node_t::~fil_node_t(),
fil_node_t::clear_backup_name(): Deal with fil_node_t::backup_name.

buf_page_t::write_fix_try(), buf_page_t::write_unfix_try(): Try to set
or unset a fake "write fix" on a page, to prevent concurrent flush()
during a backup batch. The atomic operations may run concurrently with
set_reinit() and set_freed(). The fake "write fix" does not prevent
any concurrent read or write of the page data in the buffer pool; it
only blocks writes to the underlying data file.

buf_page_t::flush(): Atomically test and set write fix, and
skip the operation if the fake "write fix" was set.

buf_page_t::set_freed(), buf_page_t::set_reinit(): Employ a
compare-and-exchange loop to accommodate for the "write fix".

innodb_backup_batch_wait(): Look up any pages that we are about to
back up. For any dirty pages, invoke buf_page_t::write_fix_try() to
try to set a fake "write fix" lock-free. If the page is currently
write-fixed between buf_page_t::flush() and
buf_page_t::write_complete(), acquire and release a page U-latch to
wait for the conflicting write to complete.

InnoDB_backup::backup_batch_start(),
InnoDB_backup::backup_batch_stop(): Adjust fil_space_t::backup_end and
fake "write fix" of dirty pages to protect the copying of a range of
pages from the underlying file.

InnoDB_backup::commit(): Enqueue the remaining log to be copied.

InnoDB_backup::checkpoint_complete(): If backup is running and
commit() has not been called, add each completed innodb_archive_log=ON
file to the queue. Else, skip or delete, as appropriate.

InnoDB_backup::archive_create(), innodb_backup_archive_create():
Hook for log_t::archive_create().

log_t::backup_start(): If we were running with innodb_log_archive=ON,
ensure that the latest file is a valid recovery starting point.
That is, wait for the latest log checkpoint to be within the file.

buf_flush_list_space(): Check for concurrent backup before writing each
page. This is inefficient, but this function may be invoked from multiple
threads concurrently, and it cannot be changed easily, especially for
fil_crypt_thread().

fil_ibd_create(): Set fil_space_t::create_lsn after the file has been
created.

dict_load_tablespaces(): Determine the size of each file if
upgrade==true. Backup depends on that.

buf_dblwr_t::begin(), buf_dblwr_t::end(), buf_dblwr_t::size():
Accessors to allow BACKUP SERVER to skip the contents of the
doublewrite buffer in the system tablespace. It is only useful for
crash recovery in case a data page had been incompletely written by
the time the server was killed. If the server is killed during a
backup, the backup will be incomplete and unusable anyway.
Furthermore, the page range locking makes page writes and backup
mutually exclusive.

Tested by: Matthias Leich
Tested by: Saahil Alam
Tested by: Rahul Raj
Co-authored by: Michael 'Monty' Widenius (mysys and ENGINE=Aria changes)
Co-authored by: Alessandro Vetere (some ENGINE=Aria changes)
Reviewed by: Thirunarayanan Balathandayuthapani (the ENGINE=InnoDB part)
Yuchen Pei
MDEV-25848 Revert SEL_TREE::keys allocation size for B-tree indexes
Kristian Nielsen
Include binlog in backup when --binlog-storage-engine=innodb

Collect the set of binlog files to include in the backup at backup commit
time, consistent with the backup LSN. Then copy the binlog files during the
FINISH stage, where locks are released; this is safe because binlog files
are append-only and recovery clears any extra data written after the backup
LSN.

This can be used to provision a slave from a backup of a master, taking the
starting GTID position from the @@gtid_binlog_pos. An mtr test case
binlog_in_engine.server_backup_slave_provision demonstrates this use case.

Signed-off-by: Kristian Nielsen <[email protected]>
Marko Mäkelä
fixup! 77311fbaf3e42d429c6c2f7ac3de1d9842b71d1b
Aleksey Midenkov
MDEV-41644 date_add_interval(): signed overflow on huge YEAR/MONTH interval values

INTERVAL::year and INTERVAL::month are unsigned, but date_add_interval()
cast them to long and did signed arithmetic before the range check. A
value above LONG_MAX became negative, and the multiplication or addition
could overflow, which is undefined behaviour. The result could also wrap
into the valid range: the year sum is stored in the 32-bit
MYSQL_TIME::year, so INTERVAL 4294967301 YEAR added 5 years instead of
failing.

The fix checks the year and month magnitude before the arithmetic, as is
already done for days with MAX_DAY_NUMBER: no interval of 10000 years or
more, or of 120000 months or more, gives a valid date.
Kristian Nielsen
Include binlog in backup when --binlog-storage-engine=innodb

Collect the set of binlog files to include in the backup at backup commit
time, consistent with the backup LSN. Then copy the binlog files during the
FINISH stage, where locks are released; this is safe because binlog files
are append-only and recovery clears any extra data written after the backup
LSN.

This can be used to provision a slave from a backup of a master, taking the
starting GTID position from the @@gtid_binlog_pos. An mtr test case
binlog_in_engine.server_backup_slave_provision demonstrates this use case.

Signed-off-by: Kristian Nielsen <[email protected]>
Rex Johnston
MDEV-42318 InnoDB range estimate doubled for ranges wider than 9 leaf pages

Prior to the MDEV-21136, the old records_in_range estimator undercounted
tall trees because it sampled only around the borders.  A compensation
doubled the estimate.

  if (root_height > divergence_height && !is_n_rows_exact)
    /* In trees whose height is > 1 our algorithm tends to underestimate:
    multiply the estimate by 2: */
    n_rows*= 2;

After MDEV-21136, this compensation is no longer needed.  This commit
removes the above lines.
Jan Lindström
MDEV-42284 fix galera_sr.mysql-wsrep#215 ER_LOCK_WAIT_TIMEOUT

Node_1's own causality wait (wsrep_sync_wait, on by default) blocked
its plain SELECT on t1 while the test held node_1's applier paused at
the sync.wsrep_apply_cb debug sync point, so the SELECT could not
reach the GTID the paused applier had not yet reached and timed out
with ER_LOCK_WAIT_TIMEOUT instead of returning.

Set wsrep_sync_wait = 0 on node_1 for that SELECT and the INSERT that
follows it, saved and restored around each of the test's three blocks.
Kristian Nielsen
Add testcase to provision a new slave from a backup of an old slave

Signed-off-by: Kristian Nielsen <[email protected]>
Aleksey Midenkov
MDEV-41643 Temporal to_string() returns a String with unset length when allocation fails

Temporal_hybrid::to_string() returned the destination String with a stale
length when String::alloc() failed, instead of signalling the failure. The
sibling to_string() methods of Interval_DDhhmmssff, Time, Datetime and Date
shared the identical flaw.

The fix returns NULL on allocation failure in all five, so they report OOM
consistently.

The val_str() implementations that return the to_string() result
directly did not set null_value, so a NULL result would be taken for a
value. They now go through the new Item::check_str(), which sets
null_value from the result, so a NULL does not stick to the item for the
next execution. The time_to_string_oom debug keyword makes
Time::to_string() fail as on out of memory.
Marko Mäkelä
squash! e3994e5a387d754120584b1c52218c2f1374527c

Remove unused my_dir_rewind()
Georgi (Joro) Kodinov
MDEV-42349: my_json_writer-t.cc fails to compile on MacOSX/clang under maintainer_mode

Converted all the stack JsonObject instances into pointers and dynamically allocated/deallocated these.
Yuchen Pei
Fix crash caused by incorrectly assigning MULTIPLE to an ARRAY key when adding an index to a table with an existing ARRAY INDEX

Missing ARRAY key type causes hliton to not be assigned in
mysql_prepare_create_table_finalize. This further causes the ARRAY key
not to be placed at the end of the key list, which it should be.

Later in TABLE_SHARE::init_from_binary_frm_image it still assumes the
last key is an hlindex key. When it is not, this non-hlindex last key
has NULL hliton, causing options() to access field of the NULL t
argument passed to it by the caller thus crash.
Marko Mäkelä
my_dir_open() wrapper for POSIX and Windows, added by Monty
Yuchen Pei
MDEV-25848 do not let get_mm_parts() match cloned key parts

get_claimed_key_part() appended clones at param->key_parts_end, so
every later get_mm_parts() call iterated over them too and cloned each
clone again. N independent predicates on the same RTREE/ARRAY field
thus produced 2^(N-1) key parts, exhausting MAX_KEY at the 8th
predicate (e.g. 8 ORed "x MEMBER OF (js)" lost range access).

Keep clones in a separate region after key_parts_end, tracked by
RANGE_OPT_PARAM::key_part_clones_end. get_mm_parts() only walks
key_parts..key_parts_end, so each predicate creates at most one clone
and N predicates use N key parts.

Assisted-By: Claude Opus 5.5 <[email protected]>
Marko Mäkelä
fixup! 7cfefd08e195bda8e03b0687632953656128ce7d
Marko Mäkelä
squash! 7cfefd08e195bda8e03b0687632953656128ce7d

Move most file-copying functions to mysys
and some file APIs to member functions of
backup_target and backup_sink.
Yuchen Pei
MDEV-25848 Range optimizer support for JSON_OVERLAPS and JSON_CONTAINS

Implemented get_mm_tree for JSON_OVERLAPS and JSON_CONTAINS.

For each JSON_OVERLAPS/JSON_CONTAINS(field, value), parse the array or
scalar element into a list. Map with get_mm_parts over the list and
tree_or/tree_and the results. At leaf level uses a helper extracted
from MEMBER OF.

Also moved JSON_OVERLAPS to inherit from Item_bool_func2_with_rev due
to its symmetry.
Sergei Golubchik
MDEV-42070 handlersocket crashes after open_index with huge number of return fields

alloca() sizes in do_exec_on_index() and cmd_find_internal() depend
on client-controlled counts (return fields, filters) bounded only by
the request line length.

* make DENA_ALLOCA_ALLOCATE always use malloc
* free key_buf/filter_buf on the early "filterblob" return

Also, a huge statement id in open_index or IN list length in exec
resized a vector to 4G elements, aborting on std::bad_alloc.
Limit both to 1024.

Assisted-by: Claude:claude-opus-5-5
Yuchen Pei
Fix crash caused by incorrectly assigning MULTIPLE to an ARRAY key when adding an index to a table with an existing ARRAY INDEX

Missing ARRAY key type causes hliton to not be assigned in
mysql_prepare_create_table_finalize. This further causes the ARRAY key
not to be placed at the end of the key list, which it should be.

Later in TABLE_SHARE::init_from_binary_frm_image it still assumes the
last key is an hlindex key. When it is not, this non-hlindex last key
has NULL hliton, causing options() to access field of the NULL t
argument passed to it by the caller thus crash.