Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
fcf83f4f1d0d...
fcf83f4f1d0d563bf88d216af45620fd4a693b65
bsrikanth-mariadb
MDEV-36523: Load basic stats from trace into optimizer

This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is controlled
by optimizer_replay_context, and points to an user defined variable name,
that has the context information.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index, cost of reading an index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records, along with their costs are also
loaded from the trace.

The trace context which is in JSON format is firstly set into the
user defined session variable, and this variable name is set to
optimizer_replay_context system variable.
Later, when a user issues a query, the contents of the json context is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.

The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.

Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
f8e0029ee007...
f8e0029ee0076644f87867df66b9787ac79a9cde
Andrei Elkin
Partial all but 3 hunks revert has_/lock/unlock- temporary_tables hunks of temporary_tables.cc.
f52bbd10c550...
f52bbd10c55058d44d879a11bdfc65431fddb30a
ParadoxV5
Merge branch 'main' into MDEV-37146
f3a5d38fa7bd...
f3a5d38fa7bd51cfff41892cc3c12fceab794964
Andrei Elkin
Revert "MDEV-37686 rpl.create_or_replace_mix2 fails in MDEV-35915 branch"

This reverts commit 3241798214b066d62ba3274ba5dc29549349ca65.
Due to MDEV-38212.
f0eb1e50ba4e...
f0eb1e50ba4ecd08bfaab2f7abffd143e60442f7
bsrikanth-mariadb
MDEV-36511: Dump basic stats of a table to trace

This feature stores the basic stats of the base tables that are used in
a query, to the optimizer trace. This feature is also controlled by
optimizer_record_context, and is not enabled by default. The stats
such as num_of_records present in the table, indexes if present
then their names, along with the average number of records_per_key
with in each index are dumped to the trace. Additionally, stats from
range analysis of the queries are also dumped into the trace

The approach taken here is to extend the existing function
store_tables_context_in_trace() and add new dump_table_stats_to_trace()
to opt_trace_ddl_info.cc. For storing the range analysis stats into the
trace, they are first recorded in the hash defined in
Optimizer_Stats_Context_Recorder, and is used later while dumping the
stats into the trace from dump_table_stats_to_trace().
Several new tests are also added to opt_trace_store_stats.test
edabb01b2865...
edabb01b28653e0fe282c6f5b8fa6642c2fb59bd
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

fts_optimize_table() : Assigns thd to transaction even it is
called via user_thread or fulltext optimize thread.
e883e0f00fbc...
e883e0f00fbc9c61ee9644af82d5556d3db73d56
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

Add FTSQueryExecutor class as a thin abstraction over QueryExecutor.
This class takes care of open, lock, read, insert, delete
for all auxiliary tables INDEX_[1..6], common FTS tables
(DELETED, DELETED_CACHE, BEING_DELETED, CONFIG..)

FTSQueryExecutor Class which has the following function:
Auxiliary table functions : insert_aux_record(), delete_aux_record(),
    read_aux(), read_aux_all()

FTS common table functions : insert_common_record(), delete_common_record(),
    delete_all_common_records(), read_all_common()

FTS CONFIG table functions : insert_config_record(), update_config_record(),
                            delete_config_record(), read_config_with_lock()
e683311e457f...
e683311e457f824e24797aa6ff4422e042b7448e
Nikita Malyavin
MDEV-38151 GTT: missing FUNCTION support

open_temporary_table[s] defaulted to searching only local temporary
tables (Tmp_table_kind::TMP). When in function, a temporary table was
"carefully re-opened": it was closed and then opened again, with
open_temporary_table. This resulted in "table not found".

The correct key in most cases is Tmp_table_kind::ANY.
In some cases it should be Tmp_table_kind::TMP:
* When operation is not supported for GTT (ANALYZE/REPAIR)
* When a global handle should be opened (CREATE VIEW)
* If it's a re-open

Apart from this bug, it caused a global temporary table to be always
opened through open_tables, meaning that tdc was queried first. This
means that the earlier code never relied on pre-opening
temporary tables.

This means that all the commands that follow the default
open_temporary_tables path in mysql_execute_command (the standard
temporary tables pre-opening) should be able to handle child GTT share.

This wasn't so for TRUNCATE, hence the changes in sql_truncate.cc.

Also, this broke the lookup order: first a local temporary table should
be looked up. If it doesn't exist, only then a global temporary table
should be looked up.
To fix the latter -- push back global temporary tables' local shares and
push front local temporary tables.

To support push_front, All_tmp_tables_list declaration is changed. It
should use I_P_List_fast_push_back adapter, which adds T **m_last to the
list body.
ca729a90a0ce...
ca729a90a0ce959e13e9963590de2eccda12487f
Nikita Malyavin
cleanup: register global_temporary_tp handlerton only ON COMMIT DELETE
ca0ca1a850a6...
ca0ca1a850a61ea15a9d35c79f4ab04ff23f774e
ParadoxV5
MDEV-28302 configurable defaults for CHANGE MASTER

Many CHANGE MASTER fields typically have the same
configurations between multi-source connections – namely:
master_connect_retry
master_ssl
master_ssl_ca
master_ssl_capath
master_ssl_cert
master_ssl_cipher
master_ssl_key
master_ssl_verify_server_cert
master_ssl_crl
master_ssl_crlpath
master_use_gtid
master_retry_count
master_heartbeat_period

When MDEV-25674 added `master_retry_count` to CHANGE MASTER, it kept
the server option `--master-retry-count` to be its default value.
This commit back-adds corresponding server options
for the defaults of the rest of those fields.
With them, the command line or config files can set up common
configurations across replication sources (and even replicas).

CHANGE MASTER can override unset
(defaulted) configurations per connection.
This commit also adds `DEFAULT` keyword support for all of those fields,
so overridden configurations can reset
to the default without RESET REPLICA.

Supporting passing the `DEFAULT` keyword also enables setting
`master_connect_retry` and `master_retry_count` to 0,
which was previously disregarded.
While here, the commit also increases `master_retry_count`
to 64-bit on LLP64 (e.g., Windows) to match LP64 (e.g., Linux).
c86b4163cce9...
c86b4163cce9c8fd1c7591285bc78bc823bf8aa8
Rucha Deodhar
MDEV-37950: INSERT ... RETURNING exposes columns for which the user lacks
SELECT privilege

Analysis:
When setup_fields() is called, the want_privilege is set to NO_ACL, so
correct priveleges are not checked.
Fix:
Since RETURNING requires SELECT_ACL privelige, when we are setting up
the returning fields for the given query, set want_privilege to SELECT_ACL.
Reset to original value of want_privilege once done.
c6f61e8d184a...
c6f61e8d184a73f2b00784740891843954c1af3e
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

Introduce QueryExecutor to perform direct InnoDB record scans
with a callback interface and consistent-read handling.
Also handles basic DML operation on clustered index of the table

Newly Added file row0query.h & row0query.cc

QueryExecutor class the following apis
read(): iterate clustered index with RecordCallback
read_by_index(): scan secondary index and fetch clustered row
lookup_clustered_record(): resolve PK from secondary rec
process_record_with_mvcc(): build version via read view
and skip deletes
insert_record(): Insert tuple into table's clustered index
select_for_update(): Lock the record which matches with search_tuple
update_record():  Update the currently selected and X-locked
clustered record.
delete_record():  Delete the clustered record identified by tuple
delete_all(): Delete all clustered records in the table
replace_record(): Tries update via select_for_update() +
  update_record(); if not found, runs insert_record.
c6120692bd01...
c6120692bd01cacbc9121d8d1063369076ad8eb1
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

Introduce AuxCompareMode and AuxRecordReader to scan FTS auxiliary
indexes with compare+process callbacks.

Replace legacy SQL-graph APIs with typed executor-based ones:
-Add fts_index_fetch_nodes(trx, index, word, user_arg,
                          FTSRecordProcessor,compare_mode).
-Redefine fts_write_node() to use FTSQueryExecutor and fts_aux_data_t.

Implement write path via delete_aux_record (or) insert_aux_record.
Keep lock-wait retry handling and memory limit checks.

Change fts_select_index{,_by_range,_by_hash} return type
from ulint to uint8_t and simplify return flow.
Include fts0exec.h in fts0priv.h and update declarations accordingly.
bd2153e8453d...
bd2153e8453db9c30bc7836b07349249b1f471a1
Marko Mäkelä
WIP MDEV-37949: Implement innodb_log_archive, innodb_lsn_archived

TODO: Make innodb_log_archive=ON writes fully work!

TODO: Rename log files and set innodb_lsn_archived on checkpoint.

TODO: Implement recovery from archived log.

The new setting SET GLOBAL innodb_log_archive=ON will enable log
archiving as soon as the current ib_logfile0 is about to wrap around.

The read-only variable innodb_lsn_archived will reflect the LSN
since when a complete InnoDB log archive is available. If it is
0 at startup, it will be recovered from the log files. If
innodb_log_archive=OFF, this variable will be adjusted to the
latest checkpoint every time a log checkpoint is executed.
If innodb_log_archive=ON, the value should not change.

When innodb_log_archive=ON, the setting SET GLOBAL innodb_log_file_size
will affect subsequently created log files when the file that is being
currently written is running out.

log_t::archive_new_write(): Create and allocate a new log file,
and write the outstanding data to both the current and the new file.

log_t::archive_new_mmap(): Create and memory-map a new log file,
and update file_size to resize_target.

log_t::archive_set_size(): Ensure that resize_target is set for
new archived log files.

log_t::first_lsn: This should be the start of the current log file,
to be consulted in log_t::write_checkpoint() when renaming files.

log_write_buf(): Add the parameter max_length, the file wrap limit.

mtr_t::finish_writer(): Specialize for innodb_log_archive=ON

innodb_log_archive_update(): Assign log_sys.archived_lsn to the latest
checkpoint; it should be usable. If the log buffer is about to wrap
around, back off and retry.

log_t::archived_mmap_switch_prepare(): Create and memory-map a new log
file. TODO: Do we need a separate function from append_prepare_wait()?

log_t::append_prepare<log_t::ARCHIVED_MMAP>(): Special case.

log_t::archived_mmap_switch_complete(): Switch to the buffer that was
created in archived_mmap_switch_prepare().

mtr_t::finish_writer(): Invoke log_close() only if innodb_log_archive=OFF.
b87a42d36459...
b87a42d36459ff9c2b0ad35a9a3291c7a4cf9190
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

Introduce ConfigReader callback to extract key, value from
fulltext config common table (CONFIG). This table has
<key, value> schema.

Simplifield all function which uses CONFIG tale. These
functions uses executor.insert_config_record(), update_config_record()
instead of SQL or query graph.
b2aa18cbada3...
b2aa18cbada304705203387a3672433b464a3b4d
Yuchen Pei
MDEV-9826 skip ci [demo] not compiling

/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x18): undefined reference to `my_hasher_base31'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x20): undefined reference to `my_hasher_crc32c'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x28): undefined reference to `my_hasher_xxh32'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x30): undefined reference to `my_hasher_xxh3'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x18): undefined reference to `my_hasher_base31'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x20): undefined reference to `my_hasher_crc32c'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x28): undefined reference to `my_hasher_xxh32'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x30): undefined reference to `my_hasher_xxh3'
collect2: error: ld returned 1 exit status
make[2]: *** [unittest/sql/CMakeFiles/explain_filename-t.dir/build.make:137: unittest/sql/explain_filename-t] Error 1
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
make[1]: *** [CMakeFiles/Makefile2:3830: unittest/sql/CMakeFiles/explain_filename-t.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
collect2: error: ld returned 1 exit status
make[2]: *** [unittest/sql/CMakeFiles/my_json_writer-t.dir/build.make:137: unittest/sql/my_json_writer-t] Error 1
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
make[1]: *** [CMakeFiles/Makefile2:3902: unittest/sql/CMakeFiles/my_json_writer-t.dir/all] Error 2
Creating mysqld link
cd /home/ycp/source/mariadb-server/main/build/sql && /usr/bin/cmake -E create_symlink mariadbd mysqld
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
[ 98%] Built target mariadbd
Creating mariabackup link
cd /home/ycp/source/mariadb-server/main/build/extra/mariabackup && /usr/bin/cmake -E create_symlink mariadb-backup mariabackup
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
[100%] Built target mariadb-backup
make[1]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
make: *** [Makefile:166: all] Error 2
af6dd561a18a...
af6dd561a18aac69c24386c0081b03b53f09ae7d
bsrikanth-mariadb
MDEV-36523: Load basic stats from trace into optimizer

This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is also controlled
by optimizer_record_context, and is not enabled by default.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records are also loaded from the trace.

The trace context which is in JSON format is firstly set into the
session variable optimizer_trace_stored_context.
Later, when a user issues a query, the contents of this variable is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.

The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.

Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
aea6a8c8caa9...
aea6a8c8caa9f74c42c0998b872e1eb016090704
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

Refactor fetch, optimize to QueryExecutor and
standardize processor API. Replaced legacy SQL-graph
paths with QueryExecutor-based reads/writes:
fts_query code now uses QueryExecutor::read(), read_by_index()
with RecordCallback (updating fts_query_match_document(),
fts_query_is_in_proximity_range(), and fts_expand_query() to call
fts_query_fetch_document() instead of fts_doc_fetch_by_doc_id(),
which was removed along with FTS_FETCH_DOC_BY_DOC_ID_* macros);

Rewrote fts_optimize_write_word() to delete (or) insert via
FTSQueryExecutor::delete_aux_record()/insert_aux_record()
using fts_aux_data_t;
9c15d10f9ab1...
9c15d10f9ab1a0102ec44e997565187103b091cc
Kristian Nielsen
Fix hang on master when disabling semi-sync

There is a global variable global_ack_signal_fd used to signal the receiver
thread to wake up when disabling semi-sync. This variable was cleared to -1
in the Ack_listener destructor, which ran at the end of the
Ack_receiver::run() function without any locking. If the thread was delayed
at that point, it could end up overwriting the new value set by a new
receiver thread. This would leave the server in a state with an invalid
global_ack_signal_fd and could cause a subsequent disable of semisync to
fail due to the wakeup not arriving at the receiver thread.

This was seen as a sporadic failure of the test case
rpl.rpl_semi_sync_cond_var_per_thd.

Fix by not modifying the global in constructor/destructor; instead set and
clear the global explicitly, allowing to clear the fd with proper locking
while the mutex is still being held.

Also fix a missing pthread_join(), which would leak thread descriptors and
allow to start a new receiver thread before the old one shut down fully.

(Either of these two changes fix the hang bug).

Signed-off-by: Kristian Nielsen <[email protected]>
7eed79cbe84a...
7eed79cbe84ad13c7119eb7ffb1f6426c5768959
ParadoxV5
aw shoot
7a34b7e56836...
7a34b7e56836ff2c759f813878ec93583d35b54c
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

- Removed fts0sql.cc file.
- Removed commented fts funtions
- Removed fts_table_t from fts_query_t and fts_optimize_t
755798d4f692...
755798d4f692e8f99677cd292a2fde66ec4d8e4e
Alexander Barkov
FETCH anchored_cursor_with_return INTO var1,var2,var3
6b656e2c74b0...
6b656e2c74b06a4cdbfc9bdfc0bce4488160a585
Yuchen Pei
MDEV-9826 skip ci [demo] not compiling

/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x18): undefined reference to `my_hasher_base31'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x20): undefined reference to `my_hasher_crc32c'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x28): undefined reference to `my_hasher_xxh32'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x30): undefined reference to `my_hasher_xxh3'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x18): undefined reference to `my_hasher_base31'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x20): undefined reference to `my_hasher_crc32c'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x28): undefined reference to `my_hasher_xxh32'
/usr/bin/ld: ../../sql/libpartition.a(ha_partition.cc.o):(.data.rel+0x30): undefined reference to `my_hasher_xxh3'
collect2: error: ld returned 1 exit status
make[2]: *** [unittest/sql/CMakeFiles/explain_filename-t.dir/build.make:137: unittest/sql/explain_filename-t] Error 1
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
make[1]: *** [CMakeFiles/Makefile2:3830: unittest/sql/CMakeFiles/explain_filename-t.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
collect2: error: ld returned 1 exit status
make[2]: *** [unittest/sql/CMakeFiles/my_json_writer-t.dir/build.make:137: unittest/sql/my_json_writer-t] Error 1
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
make[1]: *** [CMakeFiles/Makefile2:3902: unittest/sql/CMakeFiles/my_json_writer-t.dir/all] Error 2
Creating mysqld link
cd /home/ycp/source/mariadb-server/main/build/sql && /usr/bin/cmake -E create_symlink mariadbd mysqld
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
[ 98%] Built target mariadbd
Creating mariabackup link
cd /home/ycp/source/mariadb-server/main/build/extra/mariabackup && /usr/bin/cmake -E create_symlink mariadb-backup mariabackup
make[2]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
[100%] Built target mariadb-backup
make[1]: Leaving directory '/home/ycp/source/mariadb-server/main/build'
make: *** [Makefile:166: all] Error 2
64fb45093228...
64fb4509322806d24361e2f05ee46aef6ed1830e
Kristian Nielsen
Semi-sync: Clean up incorrect file/pos comparisons

The semi-sync code has a number of places where it compares pairs of
(filename,offset) for which is larger than the other.

The filename comparisons are done using strcmp(), which is wrong. Filenames
will compare wrong when they wrap from eg. bin-999999 to bin-1000000, and
user can also rename log files which can likewise break comparisons.

Further, the comparisons are completely unnecessary, as all the transactions
to be waited for are already stored in a linear list in order _and_ in a
hash table. So the code can simply use the existing hash table look and list
traversal to determine status and sequence of the waited-for transactions.

So this patch removes all the comparisons for larger/smaller, leaving only
comparisons for equality and effectively making the (filename,offset) pairs
just opaque transaction identifiers. And also removes a few other related
pieces of dead/unnecessary code.

Signed-off-by: Kristian Nielsen <[email protected]>
60e0b2abb964...
60e0b2abb964344af651e7377f60592e5a2c3c74
bsrikanth-mariadb
MDEV-36523: Load basic stats from trace into optimizer

This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is also controlled
by optimizer_record_context, and is not enabled by default.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records are also loaded from the trace.

The trace context which is in JSON format is firstly set into the
session variable optimizer_trace_stored_context.
Later, when a user issues a query, the contents of this variable is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.

The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.

Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
5a6dfe861a29...
5a6dfe861a29a684d5d613c32a35882c82ad2db8
ParadoxV5
MDEV-37530 Refactor Master & Relay Log info to iterable tuples

The persistence code of CHANGE MASTER values needs to match file
lines with the formatting for the corresponding field’s type.
This is unstructured, repetitive (not DRY), and makes feature expansions
(such as MDEV-28302 `CHANGE MASTER …=DEFAULT`)
error-prone if not difficult.

This commit moves these functions and global constants from
`slave.cc`/`.h`, as well as the Master and Relay Log Info File
entries from `Rpl_mi` and `Rpl_rli`, to dedicated `rpl_*info_file.h`
files and under corresponding structs to facilitate organization.
Namely, this commit wraps those file entries with transparent
structs that inherit from a `Persistent` interface,
or shared helper structs that themselves inherit from `Persistent`.
By moving the file read/write helper functions to or behind
implementations of `Persistent`’s virtual methods,
reading or writing the file only takes a type-agnostic loop
over the (wrapped) CHANGE MASTER values.

* This commit also includes preemptive support
  for preserving MDEV-28302’s `=DEFAULT`.
  As such, unset fields (namely `master_connect_retry`)
  now remember their `DEFAULT` states
  rather than whatever the default is at CHANGE MASTER time.
* For consistency’s sake, `master_heartbeat_period` is
  now reset at RESET REPLICA instead of CHANGE MASTER.

As this refactor will disconnect it from fixes for
some open bugs in prior versions, this commit also:
* Reimplements the value reader functions to be strict with their input
  * Fixes MDEV-38010
    number parsing ignores trailing garbage and overflows
  * Supercedes MDEV-38020 integer overflow
* Changes master_heartbeat_period from a `float` of seconds
  to a `uint32_t` of milliseconds (adding `/1000.0`s as needed)
  * Fixes MDEV-35879 `Slave_heartbeat_period` is imprecise
  * The maximum of `master_heartbeat_period` has been
    increased to 4294967.295, i.e., (2³²-1)÷1000.
  * `master_heartbeat_period` now rounds
    instead of truncates (rounds down).
* Prepares to make `master_retry_count` 64-bit on
  LLP64 (e.g., Windows) to match LP64 (e.g., Linux)
4fbc356280cd...
4fbc356280cd9cf6ceec4c1854c9ba608ec43b7c
Alexander Barkov
FETCH anchored_cursor_with_return INTO var1,var2,var3
4d8963ea4d94...
4d8963ea4d94477afe976bb6ef79cef0baa1a6ea
Yuchen Pei
MDEV-9826 More hash algorithms for PARTITION BY [LINEAR] KEY

PARTITION BY [LINEAR] KEY ALGORITHM={MYSQL51|MYSQL55|BASE31|CRC32C|XXH32|XXH3}

- The BASE31 algorithm uses a base-31 representation of the bytes, see
  Modular hashing in https://algs4.cs.princeton.edu/34hash/. It serves
  as a simple baseline
- CRC32C uses my_crc32c.
- XXH32 and XXH3 are xxhash algorithms - xxhash.h copied from latest
  release (0.8.3) of https://github.com/Cyan4973/xxHash

For performance (esp. xxh) we use one-shot hash functions in binary
hash_sort, and streaming hash function otherwise for byte-by-byte
hashing. XXH is the only stateful hash function. The other hash
algorithms are stateless and homomorphic, so streaming and one-shot
functions are identical, which is reflected in the logic of fallback
from NULL m_hash_byte to m_hash_str applied to a byte.

Tested with

mtr --suite main --do-test=.*partition
mtr --suite parts

Also ran above tests with the following patch that changes the default
algorithm from MYSQL55 to CRC32C, and XXH32 (changing the patch
accordingly)

modified  sql/ha_partition.cc
@@ -10336,6 +10336,8 @@ uint32 ha_partition::calculate_key_hash_value(Field **field_array)
  switch ((*field_array)->table->part_info->key_algorithm)
  {
  case partition_info::KEY_ALGORITHM_NONE:
+    hasher.set_algorithm(HASH_ALGORITHM_CRC32C);
+    break;
  case partition_info::KEY_ALGORITHM_55:
    /* Hasher default to mysql55 */
    break;
modified  sql/partition_info.cc
@@ -2328,7 +2328,7 @@ bool partition_info::fix_parser_data(THD *thd)
      if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
            thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
          key_algorithm == KEY_ALGORITHM_NONE)
-        key_algorithm= KEY_ALGORITHM_55;
+        key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
    }
    DBUG_RETURN(FALSE);
  }
@@ -2344,7 +2344,7 @@ bool partition_info::fix_parser_data(THD *thd)
    if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
          thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
        key_algorithm == KEY_ALGORITHM_NONE)
-      key_algorithm= KEY_ALGORITHM_55;
+      key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
  }
  defined_max_value= FALSE; // in case it already set (CREATE TABLE LIKE)
  do
modified  sql/partition_info.h
@@ -446,6 +446,8 @@ class partition_info : public DDL_LOG_STATE, public Sql_alloc
  int gen_part_type(THD *thd, String *str) const;
};

+#define PARTITION_INFO_DEFAULT_ALGORITHM partition_info::KEY_ALGORITHM_CRC32C
+
void part_type_error(THD *thd, partition_info *work_part_info,
                      const char *part_type, partition_info *tab_part_info);

modified  sql/sql_partition.cc
@@ -2471,7 +2471,7 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
  err+= str->append(STRING_WITH_LEN("KEY "));

  if (part_info->key_algorithm != partition_info::KEY_ALGORITHM_NONE &&
-      part_info->key_algorithm != partition_info::KEY_ALGORITHM_55)
+      part_info->key_algorithm != PARTITION_INFO_DEFAULT_ALGORITHM)
  {
    err+= str->append(STRING_WITH_LEN("ALGORITHM = "));
    switch (part_info->key_algorithm)
@@ -2479,6 +2479,9 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
      case partition_info::KEY_ALGORITHM_51:
        err+= str->append(STRING_WITH_LEN("MYSQL51"));
        break;
+      case partition_info::KEY_ALGORITHM_55:
+        err+= str->append(STRING_WITH_LEN("MYSQL55"));
+        break;
      case partition_info::KEY_ALGORITHM_BASE31:
        err+= str->append(STRING_WITH_LEN("BASE31"));
        break;
4432a4b2b04d...
4432a4b2b04d3754b8114505c2baeab7cce5823d
Nikita Malyavin
MDEV-38151 GTT: missing FUNCTION support

open_temporary_tables defaulted to searching only local temporary tables
(Tmp_table_kind::TMP), which is wrong -- apart from this bug, it caused
  a global temporary table to be always opened through open_tables,
  meaning that tdc was queried first.

The correct key in most cases is Tmp_table_kind::ANY.
In some cases it should be Tmp_table_kind::TMP:
* When operation is not supported for GTT (ANALYZE/REPAIR)
* When a global handle should be opened (CREATE VIEW)
* If it's a re-open

This means that all the commands that follow the default
open_temporary_tables path in mysql_execute_command (the standard
temporary tables pre-opening) should be able to handle child GTT share.

This wasn't so for TRUNCATE, hence the changes in sql_truncate.cc.

Also, this broke the lookup order: first a local temporary table should
be looked up. If it doesn't exist, only then a global temporary table
should be looked up.
To fix the latter -- push back global temporary tables' local shares and
push front local temporary tables.

To support push_front, All_tmp_tables_list declaration is changed. It
should use I_P_List_fast_push_back adapter, which adds T **m_last to the
list body.
429db8061fbb...
429db8061fbb82ba9acb819aa3525021b2bd8c02
bsrikanth-mariadb
MDEV-36523: Load basic stats from trace into optimizer

This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is also controlled
by optimizer_record_context, and is not enabled by default.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records are also loaded from the trace.

The trace context which is in JSON format is firstly set into the
session variable optimizer_trace_stored_context.
Later, when a user issues a query, the contents of this variable is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.

The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.

Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
35c3f91a1290...
35c3f91a1290ac312cfc8dc769ebb9ce78617aeb
bsrikanth-mariadb
MDEV-36523: Load basic stats from trace into optimizer

This task loads the stats of the tables that are used in a query,
from the trace into the optimizer. This feature is controlled
by optimizer_replay_context, and points to an user defined variable name,
that has the context information.
The stats such as num_of_records present in the table, indexes if present,
their names, along with the average number of records_per_key
with in each index, cost of reading an index are loaded from the trace.
Additionally, stats from range analysis i.e. ranges, and the
corresponding number of records, along with their costs are also
loaded from the trace.

The trace context which is in JSON format is firstly set into the
user defined session variable, and this variable name is set to
optimizer_replay_context system variable.
Later, when a user issues a query, the contents of the json context is
parsed and an in memory representation is built using the class
Optimizer_context_replay. This class is then used by the optimizer
to update and save original values of the tables, indexes, and range stats
in the methods "set_statistics_for_table()" of sql_statistics.cc, and
"check_quick_select()" of opt_range.cc.
After the query gets finished, the statistics that were updated
in the optimizer are restored back to the saved original values.

The entry point for parsing the json structure is in
"mysql_execute_command()" of sql_parse.cc, and similarly exit point
i.e. to restore the saved stats of the optimizer is at the end
of the same method.

Two new warning/error messages are introduced:
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_PARSE_FAILED, and
ER_JSON_OPTIMIZER_REPLAY_CONTEXT_MATCH_FAILED
2b27d18b5901...
2b27d18b5901ec1dbc46119f2220e7b3eb493cb5
ParadoxV5
Improve `mariadb-dump --dump-slave`’s desc. output

Improve the wording and spacing of the description comment generated
by `mariadb-dump --dump-slave` for its CHANGE MASTER statement.
2708cdbd4052...
2708cdbd4052a95802d99e06c82879a9842644b2
Nikita Malyavin
MDEV-38151 GTT: missing FUNCTION support
269ebb9ac752...
269ebb9ac7529ad94672784ad1f798cf29a9c8c5
Andrei Elkin
MDEV-38212 MDEV-37686 Breaks Parallel Replication

Re-fixing MDEV-37686.

NULLifying `THD::rgi_slave` pointer in `start_new_trans` ctor harmed
the parallel slave conflict detection.
Now instead of `NULL`ifying of this member a finer approach is taken to
optionally screen `THD::rgi_slave` when it is attempted to be accessed
within `start_new_trans` context, that is when such out-of-band
transaction is run by a slave thread.

The start_new_trans is allowed of course to access server local
non-replicated temporary tables, should it ever need that.

The original MDEV-37686 aspect is tested with 12.3 branch's
rpl.create_or_replace_mix2.
Any possible side effects to temporary table replication is controlled
by existing rpl suite tests.
There is no need for a specific mtr test to prove the parallel slave
side is back to normal, as `THD::rgi_slave` is again available to concurrent
transactions.
1d215e60d24e...
1d215e60d24e904d43bdb41cfa0189184a99b387
ParadoxV5
`SELECT … FROM information_schema.SLAVE_STATUS` is an alternative
to `SHOW SLAVE STATUS` that supports `WHERE` and column selection.
Utilizing this server-side feature allows `mariadb-dump --dump-slave`
to iterate only the required data on the client side.

This commit keeps support for pre-`information_schema.SLAVE_STATUS`
(MDEV-33526 @ 11.6) via `SHOW SLAVE STATUS` by switching column indices.
On the contrary, it additionally drops `--dump-slave`
and `--apply-slave-statements` support for pre-GTID &
pre-multi-source (before 10.0 – long EOL).

Reviewed-by: Brandon Nesterenko <[email protected]>
1aec47cbaaf5...
1aec47cbaaf5d515194bb15cc71375a6d5e9bb45
ParadoxV5
Bring SHOW SLAVE STATUS mode back

BTW, sounds like we are open on dropping `mariadb-dump`
support for pre-10.0. (They are long EOL.)
I’ll limit this drop to only `--dump-slave` for now.
11cda86eacb6...
11cda86eacb65a4a63e216dae97b463b9eb0c0ab
Marko Mäkelä
MDEV-37949: Introduce innodb_log_recovery_start, innodb_log_recovery_target

innodb_log_recovery_start: The checkpoint LSN to start recovery from.
This will be useful when recovering from an archived log.

innodb_log_recovery_target: The requested LSN to end recovery at.
This will be useful when recovering data files that were copied
as of a time that is before end of the available log.
0cbf3c102f50...
0cbf3c102f50f2c4253d41c7d062ee6edd288292
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB fts

Introduce CommonTableReader callback to collect doc_id_t from
fulltext common tables (DELETED, BEING_DELETED, DELETED_CACHE,
BEING_DELETED_CACHE). These table share the same schema strucutre.

Simplified all function which uses DELETED, BEING_DELETED,
DELETED_CACHE, BEING_DELETED_CACHE table. These functions
uses executor.insert_common_record(), delete_common_record(),
delete_all_common_records() instead of SQL or query graph.

fts_table_fetch_doc_ids(): Changed the signature of the function
to pass the table name instead of fts_table_t.
058fd078e153...
058fd078e153216f69eb017aea4207e56efbbd83
Yuchen Pei
MDEV-9826 More hash algorithms for PARTITION BY [LINEAR] KEY

PARTITION BY [LINEAR] KEY ALGORITHM={MYSQL51|MYSQL55|BASE31|CRC32C|XXH32|XXH3}

- The BASE31 algorithm uses a base-31 representation of the bytes, see
  Modular hashing in https://algs4.cs.princeton.edu/34hash/. It serves
  as a simple baseline
- CRC32C uses my_crc32c.
- XXH32 and XXH3 are xxhash algorithms - xxhash.h copied from latest
  release (0.8.3) of https://github.com/Cyan4973/xxHash

For performance (esp. xxh) we use one-shot hash functions in binary
hash_sort, and streaming hash function otherwise for byte-by-byte
hashing. XXH is the only stateful hash function. The other hash
algorithms are stateless and homomorphic, so streaming and one-shot
functions are identical, which is reflected in the logic of fallback
from NULL m_hash_byte to m_hash_str applied to a byte.

Tested with

mtr --suite main --do-test=.*partition
mtr --suite parts

Also ran above tests with the following patch that changes the default
algorithm from MYSQL55 to CRC32C, and XXH32 (changing the patch
accordingly)

modified  sql/ha_partition.cc
@@ -10336,6 +10336,8 @@ uint32 ha_partition::calculate_key_hash_value(Field **field_array)
  switch ((*field_array)->table->part_info->key_algorithm)
  {
  case partition_info::KEY_ALGORITHM_NONE:
+    hasher.set_algorithm(HASH_ALGORITHM_CRC32C);
+    break;
  case partition_info::KEY_ALGORITHM_55:
    /* Hasher default to mysql55 */
    break;
modified  sql/partition_info.cc
@@ -2328,7 +2328,7 @@ bool partition_info::fix_parser_data(THD *thd)
      if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
            thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
          key_algorithm == KEY_ALGORITHM_NONE)
-        key_algorithm= KEY_ALGORITHM_55;
+        key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
    }
    DBUG_RETURN(FALSE);
  }
@@ -2344,7 +2344,7 @@ bool partition_info::fix_parser_data(THD *thd)
    if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
          thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
        key_algorithm == KEY_ALGORITHM_NONE)
-      key_algorithm= KEY_ALGORITHM_55;
+      key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
  }
  defined_max_value= FALSE; // in case it already set (CREATE TABLE LIKE)
  do
modified  sql/partition_info.h
@@ -446,6 +446,8 @@ class partition_info : public DDL_LOG_STATE, public Sql_alloc
  int gen_part_type(THD *thd, String *str) const;
};

+#define PARTITION_INFO_DEFAULT_ALGORITHM partition_info::KEY_ALGORITHM_CRC32C
+
void part_type_error(THD *thd, partition_info *work_part_info,
                      const char *part_type, partition_info *tab_part_info);

modified  sql/sql_partition.cc
@@ -2471,7 +2471,7 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
  err+= str->append(STRING_WITH_LEN("KEY "));

  if (part_info->key_algorithm != partition_info::KEY_ALGORITHM_NONE &&
-      part_info->key_algorithm != partition_info::KEY_ALGORITHM_55)
+      part_info->key_algorithm != PARTITION_INFO_DEFAULT_ALGORITHM)
  {
    err+= str->append(STRING_WITH_LEN("ALGORITHM = "));
    switch (part_info->key_algorithm)
@@ -2479,6 +2479,9 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
      case partition_info::KEY_ALGORITHM_51:
        err+= str->append(STRING_WITH_LEN("MYSQL51"));
        break;
+      case partition_info::KEY_ALGORITHM_55:
+        err+= str->append(STRING_WITH_LEN("MYSQL55"));
+        break;
      case partition_info::KEY_ALGORITHM_BASE31:
        err+= str->append(STRING_WITH_LEN("BASE31"));
        break;
BuildBot (0.8.9) working for the MariaDB project.
Page built: Fri 05 Dec 2025 09:36:35 (UTC)