Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Susil
susil.behera@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41995 galera.MDEV-40727 result file has latin1 instead of utf8mb4 The four SHOW CREATE TABLE outputs in galera/r/MDEV-40727.result record DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci. That is the server default on 10.11 and 11.4, but not on 11.8, where the tables that the test creates get utf8mb4 / utf8mb4_uca1400_ai_ci. The test therefore fails on 11.8. The merge of 11.4 into 11.8 (62b26c2f7cb8) brought in the 10.11/11.4 version of the result file. Use the utf8mb4 version, which 12.3 and later already have. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39822 MHNSW index returns wrong results for M >= 128 * number of neighbors is stored in a byte * nodes have 2*M neighbors on the layer 0 => M cannot be larger than 127 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Nikita Malyavin
nikitamalyavin@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39927 Multi-key read: batch index condition pushdown for non-parametrizeable series of ranges A join that reads a local table first and looks up a remote table by key pays for the remote table in round trips or in rows. Remote means that the engine keeps its data on other machines, so every request is a network round trip, and that it can evaluate a condition next to the data. Ref access sends one request per local row. BKA sends one MRR request for all buffered keys, but the engine returns every row that shares a key, and the server filters them by the local values afterwards. In SELECT w.sym, t.id, t.ts, t.price FROM watch w STRAIGHT_JOIN trades t ON t.sym = w.sym AND t.ts >= w.since AND t.price > w.min_price with 4 local watch rows, 200 trades rows share a sym with them and 10 satisfy the ON clause: BKA returns 200 rows, where an engine that evaluated the condition would return 10. No pushdown interface allows this, because none gives the engine a condition over the local values together with the values of every buffered row. cond_push sees only the table's own columns. Under BKA the server keeps an index condition over other tables to itself and evaluates it on each fetched tuple (cache_idx_cond, skip_index_tuple). Pushdown of a select or a derived table to the handlerton would make the engine read the local table itself. MRR knows which buffered row a range answers, but a range carries only its key. Solution: the server buffers the earlier tables' rows as BKA does and hands the engine one condition, pushdown, in which their values are parameters; then, call by call, it hands over the tuples of parameter values of the buffered rows. The engine prepares pushdown once and evaluates it for each tuple next to the data. Each row it returns names its tuple by offset, and the server maps the offset back to the buffered rows and rechecks them. JOIN_CACHE_MULTI_KEY, derived from JOIN_CACHE_BKA, takes BKA's place after the join order is fixed, on ref or eq_ref access to a table whose engine sets HA_CAN_MULTI_KEY_READ, when the new optimizer_switch flag join_cache_multi_key_read, off by default, is on. EXPLAIN adds "Multi-key read" after "Using join buffer (flat, BKA join)". The interface is three handler methods in the style of index condition pushdown; MRR is left untouched: index_multi_key_read_with_pushdown_prepare(index, param, params, pushdown) once per optimization for each execution of the join: ha_index_init(index) for each call a refill of the join buffer splits into: index_multi_key_read_params(row_data) index_multi_key_read(record, &offset) until HA_ERR_END_OF_FILE ha_index_end() Design: - pushdown is a deep copy of the conjuncts of the table's condition (for an inner table of an outer join, its ON expression) that use the remote table, and otherwise only the buffered and the const tables. It keeps the ref's key equality, one top-level = or <=> on each key part the ref uses, so the engine finds the key to look up and the residual to filter by in one condition. - Parameters are copies of the buffered tables' fields in a Virtual_tmp_table, param[] is its field array, and pushdown's Item_fields point at the copies. A const table's field stays a constant: a parameter is whatever is in param[]. - A tuple is a record of that table: the parameter values only, in fixed record format, so that tuples compare byte for byte. params() gets all tuples of a call at once, as a nullptr-terminated array in a buffer the server owns; the engine reads tuple i by moving param[] by row_data[i] - row_data[0]. - The server sorts the tuples by key, merges exact duplicates and sends tuples that share a key but differ in parameters in successive calls. So each returned row answers exactly one tuple, and the engine never deduplicates. Outer joins NULL-complement the unmatched buffered rows as under BKA. The server guarantees: - no NULL key value in a tuple; - no two tuples in one call with keys the index finds equal; - param[] at row_data[0] when params() starts; - a recheck of every returned row, except the key equalities test_if_ref leaves out and an index condition the engine accepted through idx_cond_push. The engine must: - answer prepare with 0 or HA_ERR_UNSUPPORTED, or raise its own error. prepare runs during optimization, EXPLAIN included, and an acceptance lasts across executions until the next prepare or reset(). It promises no call, so pushdown applies only to the rows read() returns; - not fail ha_index_init on an accepted index: what can fail belongs in params(); - read the values before params() returns and leave param[] at row_data[0]; - return for each tuple every row that satisfies pushdown, once. The key equalities must hold exactly, since the recheck may leave them out and the split into calls relies on them; the rest of pushdown may be applied loosely, except an index condition accepted through idx_cond_push; - end a call with HA_ERR_END_OF_FILE, as any other error fails the statement, and accept ha_index_end in the middle of a call. Otherwise the join runs as plain BKA, with the same rows, decided once per plan: when a key part's value is not the ref's own (it comes from the outer query, sits only inside an OR, appears twice, or equality propagation changed it); when a nullable key value meets a part that is not null-rejecting; when pushdown holds an item with no deep copy, such as a subquery or a stored function; when a parameter is a BLOB, a TEXT, or a BIT kept among the null bits; when there is no parameter, or none comes from an engine other than the remote table's (by partition_ht(), a derived or temporary table counting as local), since there BKA's one MRR request beats a call per tuple; and when prepare returns HA_ERR_UNSUPPORTED. Limitations: - The engine must support MRR. - A partitioned remote table is not supported: ha_partition masks the flag. Tests: this commit only re-records the results that list the optimizer_switch flags; with the flag off, the join-cache tests (main.join_cache, the *_jcl6 tests, main.mrr_icp_extra, main.innodb_mrr_cpk) pass unchanged. DBUG_EXECUTE_IF points simulate out of memory where the cache and the parameter copies are created. The next commit adds the test engine TEST_MULTI_KEY. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-42002 PROXY header accepted after the first client packet The PROXY header is only valid as the very first data sent by the client. The server accepted it at any point of the connection phase, for example after the SSL request, or as a reply to an authentication switch request. Accept it only if the sequence number is 1, that is, nothing was read from the client yet. vio_reset() cleared the peer address while switching to TLS. Keep it, so the address checks after the TLS handshake see the real client. Assisted-by: Claude:claude-sonnet-5-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41737 Galera async replica SQL thread stops instead of retrying When a parallel replication worker on a Galera node loses local certification, the SQL thread stops instead of retrying the event group. - rpt_handle_event(): call wsrep_after_statement() only when the apply succeeded. On a wsrep deadlock error without a MariaDB error, raise ER_LOCK_DEADLOCK so the failure is retryable. - retry_event_group(): ignore the wsrep_after_statement() result. The error is reset right after. - Add a debug sync point and the test galera.MDEV-41737. Assisted-by: Claude:claude-4.6-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-42002 PROXY header accepted after the first client packet The PROXY header is only valid as the very first data sent by the client. The server accepted it at any point of the connection phase, for example after the SSL request, or as a reply to an authentication switch request. Accept it only if the sequence number is 1, that is, nothing was read from the client yet. vio_reset() cleared the peer address while switching to TLS. Keep it, so the real client address is not lost after the TLS handshake. After the handshake the address is only used in print_aborted_warning(). Assisted-by: Claude:claude-sonnet-5-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-42002 PROXY header accepted after the first client packet The PROXY header is only valid as the very first data sent by the client. The server accepted it at any point of the connection phase, for example after the SSL request, or as a reply to an authentication switch request. Accept it only if the sequence number is 1, that is, nothing was read from the client yet. vio_reset() cleared the peer address while switching to TLS. Keep it, so the real client address is not lost after the TLS handshake. After the handshake the address is only used in print_aborted_warning(). Assisted-by: Claude:claude-sonnet-5-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-31300 Suppress RESET SLAVE's Using_Gtid note across galera tests RESET SLAVE/RESET SLAVE ALL emits a Note when it changes Using_Gtid, and whether it does depends on the prior value, which can drift with unrelated replication changes - galera_query_cache_invalidate, galera_2_cluster and galera_gtid_2_cluster depend on it appearing. Wrap every RESET SLAVE/RESET SLAVE ALL across the galera and galera_3nodes suites in --disable_warnings/--enable_warnings, so no test depends on that value. Assisted-by: Claude:claude-4.6-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge branch '13.1' into 13.2 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41562: Old slave with semi-sync and binlog-in-engine master With binlog-in-engine, semi-sync replication is supported only when the slave (and the master) is at least MariaDB 13.2. So make sure to give an error if an old slave connects to a master with binlog-in-engine enabled and the slave has semi-sync enabled. Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| new columnstore 25.10.8 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Ahmad
ahmedshapan913@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-27837: Disallow SET @@session.server_id within transaction Setting @@session.server_id within a transaction is dangerous for replication as it can cause inconsistencies in how events are logged and replicated. This patch adds a check in the server_id system variable's ON_CHECK handler to disallow changing @@session.server_id inside an active transaction or substatement (triggers, stored procedures). Setting @@global.server_id inside a transaction remains allowed since it does not affect the session's binlog state. The check uses the generic ER_CANT_SET_IN_TRANSACTION and ER_CANT_SET_IN_SUBSTATEMENT error codes, consistent with how gtid_domain_id handles the same restriction. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
bsrikanth-mariadb
srikanth.bondalapati@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41862: Syntax error with DELETE and IN in FederatedX pushdown A single-table DELETE with an IN subquery on the same table, pushed down to FederatedX, failed on the remote server with a syntax error: DELETE FROM t3 WHERE a IN (SELECT a FROM t3 WHERE a < 3); The target table of a single-table DELETE is not in top_join_list, so the list is empty. st_select_lex::print() printed it as "delete from using dual where ...". Fix: in st_select_lex::print() use top_join_list only when it is not empty, and otherwise print the tables from leaf_tables. For DELETE that branch emits the "from " keyword. UPDATE is unchanged. As a side effect, EXPLAIN EXTENDED of a single-table DELETE no longer shows the dummy "using dual" and prints "delete from `t` where ...". A DELETE through a view now prints the view instead of its base table. Update the affected results (explain_non_select, myisam_explain_non_select_all, opt_hints, opt_hints_index, opt_trace, versioning.delete_history). Add a test to federatedx_pushdown_upd_del that pushes down DELETE (also with ORDER BY ... LIMIT and as a prepared statement) and UPDATE with an IN subquery on the same table, and checks both the resulting data and the statements received by the remote server in its general log. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41395 fil_space_t::create() may release fil_system.mutex If fil_space_t::create() ends up releasing fil_system.mutex for invoking fil_crypt_threads_signal(), another thread may grab fil_system.mutex and find a tablespace that contains no data files. Let us fix this race condition and remove some checks that now are redundant. By default fil_crypt_must_default_encrypt() holds, and then fil_crypt_thread() waits indefinitely in wait_for_work() once the default_encrypt list has been exhausted. A tablespace for which no fil_crypt_threads_signal() is issued would therefore never be encrypted at all. fil_space_t::create(): Do not invoke fil_crypt_threads_init(); let the caller do that. During database startup, before fil_crypt_threads_init() has been invoked, there is no point to invoke this function; the encryption queues will be properly set up for the tablespaces that have been opened by that time. fil_ibd_create(): Invoke fil_crypt_threads_signal() after invoking fil_space_t::clear_stopped() and releasing fil_system.mutex. fil_crypt_threads_signal(): Check that fil_crypt_threads_init() has been called. The function dict_table_open_on_name() will be invoked during early server startup. dict_table_open_on_name(): Invoke fil_crypt_threads_init() at the end if the table was loaded into the cache and the tablespace likely as well. This should be the main code path through which old tablespaces are opened via SQL. dict_load_tablespace(): Invoke fil_crypt_threads_signal(), because fil_ibd_open() may have added the tablespace to fil_system.default_encrypt_tables. This is the single point through which every dictionary-driven tablespace open passes; it is only invoked by dict_load_table_one(), hence by dict_sys_t::load_table(). We will not invoke fil_crypt_threads_init() in dict_load_table_on_id() or dict_sys.load_table(); it will instead be invoked at a higher level, typically when a table definition is first loaded by name, in dict_table_open_on_name(). buf_dump_load_func(): After the non-first buf_load(), invoke fil_crypt_threads_signal() in order to start the encryption of any newly loaded tablespaces. drop_garbage_tables_after_restore(), wsrep_append_foreign_key(), ha_innobase::create(), ha_innobase::truncate(), row_rename_table_for_mysql(), i_s_sys_tables_fill_table(), i_s_sys_tables_fill_table_stats(): Invoke fil_crypt_threads_signal() in case some tablespace metadata was loaded. row_import_cleanup(): Invoke fil_crypt_threads_signal() to register the imported tablespace. i_s_sys_tablespaces_fill_table(), fil_system_t::find(), Datafile::validate_first_page(), fil_space_t::name(), fil_space_t::read_page0(), fil_space_t::prepare_acquired(), fil_space_t::try_to_close(), fil_crypt_default_encrypt_tables_fill(), fil_system_t::default_encrypt_next(): Remove a now-redundant check for a tablespace that contains no files. srv_n_fil_crypt_threads_started: Declare as Atomic_relaxed<uint>, for the unlocked read in fil_crypt_threads_signal(). Co-developed by: Thirunarayanan Balathandayuthapani |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge 13.1 into main | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-38741 Abort the running statement when its client disconnects The server notices that a client is gone only when it reads from or writes to the socket. While a statement runs, for example waiting for an InnoDB row lock, a disconnect goes unnoticed. The statement keeps running, and its transaction keeps owning its locks, so other connections wait for locks of a client that is not there. Watch the client socket from the end of login until the connection ends, or the client sends COM_QUIT, using only the events the OS reports on its own. When the client has gone, abort the running statement with the new KILL_DISCONNECT state, found by thread id like KILL does, through a kill_thd_locked() that KILL shares. A quitting connection ignores it. KILL_DISCONNECT is statement-level, like KILL QUERY. When it does abort a statement, the connection ends after the error of that statement is sent, so the commands that the client pipelined behind it do not run. A disconnect noticed while the connection is idle, before the statement starts, is remembered and applied when the next statement starts. The reason, "Client disconnected", is logged once, and a failed write of the error is not reported. An orderly COM_QUIT, or a close without it, is handled as before. The global variable disconnect_watcher, ON by default, can be set OFF to switch the watcher off. It affects the connections made after the change, and exists only where there is a watcher: Linux, Windows and the BSD systems, not the embedded server. If the watcher cannot start, the variable is OFF and cannot be set ON. Linux: an epoll set, polled by the connection accepting loop. EPOLLRDHUP is requested, as a close and a crashed client can be only a FIN. Windows sockets: WSAEventSelect(FD_CLOSE), with a wait per socket on a thread pool of our own that has a single thread, so the callbacks run one by one. Windows named pipes: the wait is for the exit of the local client process. A client that only closes the pipe is not noticed. BSD and macOS: kqueue, EV_EOF. Replication dump threads are watched too. mysqltest: dirty_close takes an optional "linger" (the default, a FIN) or "nolinger" (SO_LINGER 0, so the server gets a reset) argument. main.disconnect_lock_wait, disconnect_lock_owner (the client's transaction is rolled back and its locks are released), disconnect_wait_kinds (a CPU-bound statement and a table level lock wait, which do not poll the connection themselves), disconnect_lock_wait_idle (debug), disconnect_lock_wait_pipe (Windows) and sys_vars.disconnect_watcher_basic are new. mysql_client_test gets test_disconnect_pipelined: a client sends two commands in one write and half-closes while the first waits for a lock. The connection ends, and the second command is not run. disconnect_lock_wait and disconnect_lock_wait_pipe run with the watcher ON and OFF, the others need it ON. innodb.xa_recovery no longer needs the shortened lock wait timeout with the watcher ON. rpl_ssl1 and rpl_password_boundaries wait for the session of the slave to end before DROP USER, max_binlog_total_size and rpl_fail_register kill the dump thread only if it is still there, and rpl_get_lock suppresses the read error of a reset connection on Windows (errno 10054). Assisted-By: Claude:claude-sonnet-5-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
compilation failure storage/innobase/row/row0import.cc:678:17: error: conversion from 'uint32_t' {aka 'unsigned int'} to 'uint16_t' {aka 'short unsigned int'} may change value [-Werror=conversion] 678 | ? 0 : FSP_FLAGS_GET_ZIP_SSIZE(m_space_flags); |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Implement support for semi-sync with --binlog-storage-engine Call report_binlog_update() when semi-synchronous replication is enabled and using --binlog-storage-engine. Only AFTER_COMMIT is available. The AFTER_SYNC in legacy binlog has the property that changes in a committing transaction does not get visible to other transactions until the slave has acknowledged, avoiding phantom reads if the master fails permanently just after. This requires the two-phase commit between binlog and storage engine so that the binlog is written before the transaction is engine-committed. However, the whole point of --binlog-storage-engine is to avoid the expensive two-phase commit. (The ability to avoid phantom reads could be later implemented as an AFTER_PREPARE option, which would send binlog to slave and await ack before it is written/committed into the engine). Some existing semi-synchronous replication tests are adapted to also run in the binlog_in_engine suite. Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Nikita Malyavin
nikitamalyavin@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39927 Multi-key read: test engine TEST_MULTI_KEY No engine in the server tree implements the multi-key read, so no test in the tree takes the path. TEST_MULTI_KEY is a MEMORY engine that does: for each tuple, its read looks up the tuple's key and returns the rows the pushed condition admits. Session variables count its prepares, calls, tuples and rows. Its suite test_multi_key runs each query with join_cache_multi_key_read off, through SET STATEMENT, and then on, and shows the counters and the EXPLAIN of the path. The case with a ? takes the path because of MDEV-41238, which gives a bound ? a copy. ha_heap is no longer final and is exported from mariadbd, as ha_maria is, so that the dynamic plugin can derive from it. Windows does not build the engine: plugins there link against server.dll, which does not export ha_heap. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Semi-sync: Implement support for using GTID in semi-sync ACKs Implement the necessary logic in class Repl_semi_sync_master_gtid and related code, so that the semi-sync master can request the slave to put the GTID in the reply ACK packet, instead of the filename/offset. In GTID mode, when a slave connects (which it should do using a GTID start position), the latest GTID in the starting position (if any), as determined by the list of transactions pending acks, is used as the point at which to implicit ACK anything pending. The semi-sync logic is otherwise unchanged, in GTID mode it just uses the GTID as the transaction identifier instead of the file/pos of the end of the event group. This patch only enables the GTID-based semisync for binlog-in-engine, but the mode works with old binlog implementation as well, and passes all tests (if enabled by code change). Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41860 return vector extended metadata This adds "vector" as an extended metadata type for when is returning a collection of 4 byte little endian floats from teh server. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge 11.8 into 12.3 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Semi-sync: Clean up incorrect file/pos comparisons The semi-sync code has a number of places where it compares pairs of (filename,offset) for which is larger than the other. The filename comparisons are done using strcmp(), which is wrong. Filenames will compare wrong when they wrap from eg. bin-999999 to bin-1000000, and user can also rename log files which can likewise break comparisons. Further, the comparisons are completely unnecessary, as all the transactions to be waited for are already stored in a linear list in order _and_ in a hash table. So the code can simply use the existing hash table look and list traversal to determine status and sequence of the waited-for transactions. So this patch removes all the comparisons for larger/smaller, leaving only comparisons for equality and effectively making the (filename,offset) pairs just opaque transaction identifiers. And also removes a few other related pieces of dead/unnecessary code. Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Alexey Botchkov
holyfoot@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41430 auth_pam_tool leaves zombie processes. As the auth_pam_tool claims root privileges, the mariadbd can't kill it if it hangs. So instead of sending the SIGKILL to the auth_pam_tool we set the alarm() there so it kills itself after the timeout. The alarm counter is hardcoded to 100 seconds so the authentication can be halted even if the connect_timeout allows it to be longer. The auth_pam.c now calls the waitpid() without the WNOHANG, so it waits for the auth_pam_tool to return. If the PAM module aggressively resets the alarm or handles the SIGALARM the pam_auth() can hang forever. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-42002 PROXY header accepted after the first client packet The PROXY header is only valid as the very first data sent by the client. The server accepted it at any point of the connection phase, for example after the SSL request, or as a reply to an authentication switch request. Accept it only if the sequence number is 1, that is, nothing was read from the client yet. vio_reset() cleared the peer address while switching to TLS. Keep it, so the real client address is not lost after the TLS handshake. After the handshake the address is only used in print_aborted_warning(). Assisted-by: Claude:claude-sonnet-5-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Semi-sync: Refactor in preparation for using GTID in semi-sync acks This is a refactor patch that contains no/little logic changes but a lot of mostly mechanic code changes to prepare for allowing to use either old-style filename/offset or new-style GTID to identify an event group in the semi-sync ack. The idea is to replace all explicit filename/offset function arguments with a generic Repl_semi_sync_trx_info *inf to identify an event group (aka "transaction"). This object can then be used to look up in the semi-sync hash table by either file/pos or by GTID. The classes Active_tranx and Repl_semi_sync_master are sub-classed into Active_tranx_file_pos/Active_tranx_gtid and Repl_semi_sync_master_file_pos/Repl_semi_sync_master_gtid. Virtual functions are implemented in each for comparing identifiers and for calculating hash keys, using either the file/pos or the GTID as appropriate. This way, the existing logic can now be used with either (only file/pos is actually used in this patch; adding GTID is for a subsequent patch). For the binlog writing side, report_binlog_update(), wait_after_sync(), and THD::semisync_info are extended to also take the GTID of the event group, which is already available in the calling code. For the dump thread / slave connection side, update_sync_header() is extended to take also the GTID. The dump thread code is extended to keep track of the GTID of the current event group (slightly extending the logic already there to keep track of event groups). Also, the dump thread now only passes the last event of an event group into the semisync layer (the other events are redundant, as they are never semi-sync ack'ed, saving needless semisync locking and hash lookups). Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge 12.3 into 13.1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41491 parsec ASAN crash on manually modified mysql.global_priv table the server didn't invoke auth plugin's preprocess_hash() for empty passwords, thus it didn't check whether empty passwords were acceptable and plugin's auth() method reliance on the binary hash being correctly preprocessed was broken. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Kristian Nielsen
knielsen@knielsen-hq.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Fix sys_vars.tmp_disk_table_size_basic when no perfschema Signed-off-by: Kristian Nielsen <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Jan Lindström
jan.lindstrom@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-31300 Suppress RESET SLAVE's Using_Gtid note across galera tests RESET SLAVE/RESET SLAVE ALL emits a Note when it changes Using_Gtid, and whether it does depends on the prior value, which can drift with unrelated replication changes - galera_query_cache_invalidate, galera_2_cluster and galera_gtid_2_cluster depend on it appearing. Wrap every RESET SLAVE/RESET SLAVE ALL across the galera and galera_3nodes suites in --disable_warnings/--enable_warnings, so no test depends on that value. Assisted-by: Claude:claude-4.6-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-42002 PROXY header accepted after the first client packet The PROXY header is only valid as the very first data sent by the client. The server accepted it at any point of the connection phase, for example after the SSL request, or as a reply to an authentication switch request. Accept it only if the sequence number is 1, that is, nothing was read from the client yet. vio_reset() cleared the peer address while switching to TLS. Keep it, so the real client address is not lost after the TLS handshake. After the handshake the address is only used in print_aborted_warning(). Assisted-by: Claude:claude-sonnet-5-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Merge 11.4 into 11.8 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-39492 PQ: cost the parallel scan in the join-order search Until now the join order was chosen by the serial optimizer and the driving table it settled on was then handed to the workers whenever the gate admitted the query. On TPC-H that split REGION in Q5 and NATION in Q11: five- and twenty-five-row tables of which one row survives the WHERE, so the whole join under them ran on one worker and the query paid the setup of the workers for nothing. The plan that would actually divide -- ORDERS first in Q5, split four ways with the lookups below it spread evenly -- costs 1917 against the chosen plan's 873 when read serially, and the serial optimizer had no way to know that read by four workers it costs about 490. best_access_path() now prices the driving table's scan both ways and keeps the cheaper (parallel_scan_cost): serially, or as one worker's share of the scan plus optimizer_parallel_setup_cost per table in the join, with the rows surviving the table's own WHERE spread over min(N, rows_out) workers. The join-order search divides every later position's cost by that divisor (position_cost_in_plan), since each worker joins only its share of the driving rows and the plan's cost is one worker's. A large table can therefore win the driving position it loses serially, while a tiny driver gains nothing from the division and is charged the setup, so it stays serial. parallel_join_check() runs the workers only for a plan chosen that way. Three things are needed for the search to find such a plan honestly. The pruning heuristic drops a prefix that is worse on both rows and cost than the best at its depth, which is always true of a big-table prefix next to a five-row one; prefixes under a parallel driver are now compared only with each other. The tables under a parallel driver are costed as a worker runs them, a plain nested loop with no join buffer, hash join or rowid filter, and no covering-index or rowid-filtered scan is offered as a driver, because the gate refuses all of these and a plan chosen on their cost would run serially. And the refusals the gate can make without a plan -- limits, DISTINCT, HAVING, outer joins, window functions, aggregates whose partials do not merge, a GROUP BY on an expression, a select-list column GROUP BY does not name, an item a worker cannot evaluate -- are asked before the search as well (pwt_query_shape_allows_workers), so that a query the workers will refuse is costed as the serial plan it will run. Q7's GROUP BY on extract(year ...) is the case that showed why: chosen for LINEITEM first on the discount, refused after the plan, and run serially in that shape at five times its serial time. The setup cost is a session variable, optimizer_parallel_setup_cost, in microseconds, default 2000 per table. It was measured on this branch at about 2 ms for a single-table scan and about 8 ms for a three-table join, dominated by creating the worker threads with a THD each, opening every table once per worker, and building and draining the result containers. The parallel-query tests set it to 0, since their tables are far too small to repay it, and main.parallel_query_costing pins the three behaviours on a five-row dimension and a 20000-row fact table. Measured on TPC-H SF1 with histograms, four workers against serial: Q5 490 -> 164 ms, Q3 623 -> 188 ms, Q10 341 -> 139 ms, Q12 615 -> 191 ms, Q15 1225 -> 396 ms, Q1 3966 -> 1160 ms; Q11 and Q7 unchanged. This commit was prepared with Claude Code (Fable 5.1), which diagnosed the REGION/NATION splits and the Q7 regression on a scratch copy of the TPC-H data, implemented the costing, the two-family pruning, the worker-shaped costing of the parallel chain and the pre-plan gate, wrote the costing test, updated the existing tests, and ran the parallel-query tests in both protocols and the main suite. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||