Console View
|
Categories: connectors experimental galera main |
|
| connectors | experimental | galera | main | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Marko Mäkelä
marko.makela@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Interleaved read-ahead in pread_write | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40287 AES_ENCRYPT() and KDF() return NULL with OpenSSL 4.0 MyCTX used an EVP_CIPHER_CTX in a stack buffer instead of allocating it with EVP_CIPHER_CTX_new(). OpenSSL 4.0 rejects a context that was not created that way for ciphers that use an IV: EVP_CipherInit_ex() fails with "invalid iv length", so AES_ENCRYPT() and KDF() in CBC/CTR/GCM modes return NULL. ECB has no IV and still works, which is why only the non-ECB modes broke. Allocate the context with EVP_CIPHER_CTX_new()/EVP_CIPHER_CTX_free(), and on an init error free and null it, so a later update()/finish() from a caller that ignored the error is a safe no-op, not a double free. Remove check_openssl_compatibility() and the EVP_CIPHER_CTX_SIZE and EVP_CIPHER_CTX_init macros. my_md5() used the same stack-buffer-sized-by-EVP_MD_CTX_SIZE hack, whose only guard was check_openssl_compatibility(). MD5_CTX is a concrete struct in both OpenSSL and WolfSSL, so use it directly with MD5_Init/Update/Final on a caller-owned context, like my_sha.inl does for SHA*_CTX; this removes the EVP_MD_CTX hack and EVP_MD_CTX_SIZE. The exception is OpenSSL 1.0.x, which aborts on the low-level MD5 API in FIPS mode (MDEV-7788): there, and only there, keep the EVP path with the non-FIPS flag. 1.1.0 removed the FIPS Object Module and 3.0+ low-level MD5 bypasses the provider, so the concrete MD5_CTX is correct on every other version. SHA is unaffected, it already used concrete SHA*_CTX structs. Verified against OpenSSL 4.0.1: the mysys aes-t test fails on the CBC/CTR/GCM cases with the stack buffer and passes with EVP_CIPHER_CTX_new(). The 1.0.x MD5 path verified against a FIPS build of OpenSSL 1.0.2u: the EVP path computes MD5, the low-level path aborts. No visible performance degradation: the extra allocation costs ~12 ns/call (WolfSSL) and ~30 ns (OpenSSL) on Windows at a 30-byte payload, nothing at 16 KB, and nothing on Linux/glibc; sysbench OLTP over encrypted tables and redo log is unchanged. Tests main.tlsv13 and main.ssl_crl: OpenSSL 4.0.3 reports TLS alerts as "tls alert ..." (was "ssl/tls alert ..."), accept both. Assisted-by: Claude:claude-opus-4-8 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
PranavKTiwari
pranav.tiwari@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25515 User Account Host Names using CIDR notation CREATE USER u@'192.168.0.0/24'; -- same as '192.168.0.0/255.255.255.0' CIDR is an input spelling only, normalised to the netmask form while parsing. A malformed mask is now rejected instead of stored, and one already in the privilege tables is skipped at load with a warning instead of being honoured. Nothing rejected these before, and they are not merely useless: a non-contiguous mask such as 10.0.0.0/255.0.255.0 matches the scattered set 10.*0.*. Skipping rather than erroring keeps such a row droppable and renamable, except for a literal CIDR host stored by an older version (for example 10.0.0.0/24): the parser converts that name, so it can no longer be named and must be removed with a direct DELETE from the privilege tables. New error ER_INVALID_HOST_NETMASK, ER_ACL_ENTRY_IGNORED_BAD_HOST_MASK |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oleksandr Byelkin
sanja@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-34817 Performance schema is not cleared of removed package routines Package routines are recorded in performance schema as PROCEDURE/FUNCTION "<package>.<routine>", but dropping a package body removed only the PACKAGE BODY record, so routine statistics survived DROP PACKAGE [BODY] and DROP DATABASE. Drop the routine records of the package together with the package body record. Also reset statistics of a newly allocated PFS_program (as in 089caf901f9), so a reused slot does not inherit stale counters. Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 Extract vers_set_starts() and add starts_clause Extracts the STARTS-alignment logic into vers_set_starts() so it can be reused. If the aligned local time falls into a DST gap (e.g. a midnight that does not exist), the start of the gap is used instead of failing with ER_PART_WRONG_VALUE. If it falls before the TIMESTAMP range (e.g. a local midnight before 1970), the timestamp itself is used. Also adds the Vers_part_info::starts_clause flag in preparation for later commits. The flag records whether STARTS was given explicitly (by the user or in the unpacked frm), which the auto-partitioning code uses to decide whether to re-align STARTS to the existing history. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40012 PQ: refuse a plan that reads a table through a join buffer A worker has no join buffer. setup_worker_jointabs() stripped the cache off every worker JOIN_TAB and joined a row at a time, and pwt_table_conds() handed the worker the buffer's scan filter (cache_select->cond) as well as select_cond so that it still rejected what the buffer would have. The result was correct, but it was not the plan the optimizer costed: BNL, BNLH, BKA and BKAH were chosen for scanning the inner table once per buffer-full of outer rows, and a worker scanned it once per driving row instead. For a full-scan inner table that can be orders of magnitude more work than the serial plan the parallel one is meant to beat. So the gate now refuses the plan, as the prototype did. The check sits in can_run_query_in_workers() beside the other per-table refusals, on tab->cache or tab->use_join_cache, both of which make_join_readinfo() has settled by the time parallel_join_check() runs in optimize_stage2(). The refusal names the table in the optimizer trace. The worker-side handling of a buffered table goes with it: - pwt_table_conds() becomes pwt_table_cond() and no longer reports cache_select->cond. remove_redundant_bnl_scan_conds() only moves conjuncts out of select_cond when the tab has a BNL/BNLH cache, so for every tab the gate now admits select_cond is already the whole condition. A tab can still carry a cache_select without a buffer, because make_join_select() builds the scan filter before check_join_cache_usage() decides, but its cond is then a copy of conjuncts select_cond still holds. The gate no longer asks whether that copy is worker-safe, since no worker evaluates it. - pwt_clone_table_conds(), which cloned both halves and ANDed them, is gone; setup_worker_jointabs() clones the one condition with pwt_clone_rebind(). - setup_worker_jointabs() no longer clears cache, use_join_cache and jbuf_tracker on the worker's copy; pwt_assert_tab_inert() asserts that the manager's tab has no cache instead. The copy's cache_select is still cleared, as it can be set on a tab without a buffer. parallel_query_join: the BNL case now expects the query to run serially, with the same rows. The full-scan inner table case (d3) was getting a BNL buffer under the default join_cache_level, so it runs with join_cache_level=0 to keep covering a worker-side full scan of an inner table. parallel_query_why pins the new refusal reason. This commit was prepared with Claude Code (Opus 5.5), which wrote the gate check, removed the worker-side join buffer handling it made unreachable, updated the tests, reconfigured and rebuilt the worktree, and ran the parallel_query tests in normal and --ps-protocol modes. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40799 Runtime plugin/UDF load errors lost under --silent-startup Regression from MDEV-32745 (7828fb475b0), which guarded the plugin-load my_error() calls with opt_silent_startup. That option is a lifetime global, set once at startup and never reset, so the guard suppressed the SQL error for the whole server lifetime, not just during startup. Runtime operations (INSTALL PLUGIN, CREATE FUNCTION ... SONAME) then skipped my_error(), never set the diagnostics area and wrongly succeeded - e.g. main.ps's "call proc_1()" no longer failed with ER_CANT_OPEN_LIBRARY. Startup callers pass MYF(ME_ERROR_LOG); runtime callers pass MYF(0). Gate the silencing on that flag via silent_plugin_startup() so it applies only to the startup error-log path, and runtime errors always reach the client. No new test case: the runtime failure path is already covered by existing tests (e.g. main.ps's ER_CANT_OPEN_LIBRARY check). The regression stayed invisible only because stock MTR does not start servers with --silent-startup. A dedicated test would have to restart the server with --silent-startup solely to assert that a startup-only option does not affect runtime, which adds little over the restored invariant. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41228 Remove the deep copy debug hooks pushdown_copy(), formerly deep_copy(), is left with three callers outside condition pushdown: - Virtual_column_info::clone(), which has had no callers since ef5adf52076 (MDEV-24274), - the "item_cache_clones" debug hook in Item::get_cache(), and - the "split_materialized_clones" debug hook in TABLE::add_splitting_info_for_key_field(), which used a copy of key_field->val for the lookup into the split-materialized table. The debug hooks were added only to exercise deep_copy() on Item_cache items, together with Item::check_deep_copy(), which compared a deep copy with its original. Split materialization no longer copies with pushdown_copy(): the left side of the pushed equality uses same_select_copy() and the right side new_select_copy(). Remove all of them, with check_deep_copy(). Item::get_cache() is inline again, as it was before the debug hook. main.item_clone_debug ran its query under the split_materialized_clones flag and is removed; the same query, with three rows in t0 so that a stale value shows in the result, is in main.derived_split_innodb. Item_cache::pushdown_copy_cache(), the former deep_copy_cache(), was only called from Item_cache::pushdown_copy(). Fold it into that override. This commit was prepared with Claude Code (Opus 5), which removed the debug hooks, check_deep_copy() and Virtual_column_info::clone(), restored the inline Item::get_cache(), removed main.item_clone_debug, folded Item_cache::pushdown_copy_cache() into Item_cache::pushdown_copy(), and ran main and the SQL-layer suites in normal and --ps-protocol mode. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 Replace COMBINE macro with interval2sec() and interval2usec() Turn the COMBINE macro of date_add_interval() into inline functions with overloads for INTERVAL and MYSQL_TIME, so that the history partition sizing can reuse the seconds part. Move the "Daynumber" comment that sat above the macro to MAX_DAY_NUMBER, which it describes. MAX_DAY_NUMBER moves to sql_time.h as a constexpr (the unused copy in item_timefunc.cc is removed), and the range check of date_add_interval() becomes the inline interval_days_out_of_range(), so that the sizing can check that the seconds do not overflow. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-36295 INTERVAL function returns inconsistent results when used in a WHERE clause A query like select * from (select a, b from t1 group by a, b) d where interval(d.a, 3, 7) = 1 returned no rows when the condition was pushed into the derived table, although the row (5,20) matches, as condition_pushdown_for_derived=off shows. Item_func_interval kept its first argument, the Item_row, in a member of its own as well as in args[0]. Condition pushdown copies the condition with deep_copy(). The copy of the node got new arguments but kept the member, so it went on reading the original row: the INTERVAL() pushed into the derived table compared the original outer column, and the pushed condition rejected every row. Item_func_interval now reads the row through args[0] alone, so a copy cannot point anywhere else. This commit was prepared with Claude Code (Opus 5), which found the stale member, made Item_func_interval read the row through args[0], wrote the test and confirmed in a build of the parent commit that it fails without the change, and ran main and the SQL-layer suites in normal and --ps-protocol mode on the series that contains it. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
forkfun
alice.sherepa@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-24557 mariadb-dump: import MySQL 8.x users, grants and roles mariadb-dump --system=users/all now dumps a MySQL 8.0+ source as MariaDB statements. A MySQL source is detected by mysql.role_edges, not by version (MySQL 26.x is numbered above MariaDB). MySQL source: - Privileges are mapped to MariaDB ones where possible (BINLOG_ADMIN -> BINLOG ADMIN, BINLOG REPLAY). Others, including the ones whose nearest MariaDB privilege is wider, are dropped with a # WARNING. - CREATE USER clauses without an equivalent are dropped. An auth plugin MariaDB lacks drops IDENTIFIED and locks the account. Authentication factors after the first are dropped with a warning. - The GRANT ... AS clause and partial REVOKEs are dropped with a warning, so the imported grant is wider than on the source. - Roles are found by the shape CREATE ROLE leaves in mysql.user or by being granted onward. Same-named roles on several hosts merge into one. An account that is both a user and a role is dumped as both. - Role grants go to the bare role name, so no login account is created. - Only the first of several default roles is kept, the rest are warned. Both sources: - Statements containing "*/" are refused with a warning: it would end the /*M!... */ wrapper and run the rest as SQL on import. - The import sets NO_AUTO_CREATE_USER, so a GRANT to a missing account fails instead of creating a passwordless login. - CREATE ROLE is emitted once per role, not once per admin row. - The self-DROP guard compared current_user() with a quoted name and never matched; it now uses the raw user@host. - DROP USER moves to /*M!100103, MESSAGE_TEXT is a fixed string, and the MySQL-only /*!80001 statements are gone. The translators are in client/mysql8_translate.cc with a unit test on captured MySQL output. Assisted-By: Claude:claude-5.5-sonnet |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 Comment on partition_element allocation in set_up_default_partitions() partition_element and its push_back() list node are allocated on the active arena (thd->mem_root). The allocation failure is already reported there (by my_malloc() or the arena's error handler), which is why the failure path only returns the error. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 Fix to_string() OOM return semantics for temporal types Temporal_hybrid::to_string() returned the destination String with a stale length when String::alloc() failed, instead of signalling the failure. Return NULL on allocation failure. Apply the same fix to the sibling to_string() methods of Interval_DDhhmmssff, Time, Datetime and Date, which shared the identical flaw, so all five report OOM consistently. The val_str() implementations that return the to_string() result directly did not set null_value, so a NULL result would be taken for a value. They now go through the new Item::check_str(), which sets null_value from the result, so a NULL does not stick to the item for the next execution. The time_to_string_oom debug keyword makes Time::to_string() fail as on out of memory. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 Auto-create: Pre-existing historical data is not partitioned as specified by ALTER Adds logic into prep_alter_part_table() for INTERVAL ... AUTO to read the history range (vers_get_history_range()) and size the new layout from it. STARTS is aligned to the oldest history row_end (vers_set_starts()) if it was not specified by the user, or if the user specified it later than that; in the latter case WARN_VERS_WRONG_STARTS is printed. The history partitions cover both the newest history row_end and the query time, so DML after the ALTER does not need to create partitions. A PARTITIONS N clause is kept if it gives more partitions. An explicit subpartition list is replaced by as many default subpartitions. If the layout needs more than MAX_PARTITIONS partitions (counting subpartitions), or the INTERVAL is longer than the date range, the ALTER fails with ER_VERS_HIST_PART_FAILED. Month and year intervals are stepped in UTC; a boundary past the maximum TIMESTAMP ends a partial last history partition. In case of fast ALTER TABLE, f.ex. when partitioning already exists, the above logic is ignored unless FORCE clause is specified. When user specifies partition list explicitly the above logic is ignored even with FORCE clause. With FORCE on a table already partitioned by SYSTEM_TIME the work is done on a clone of the partitioning info. The parsed settings are copied into the clone's vers_info with Vers_part_info::copy_settings(), which keeps the clone's now_part and hist_part (the parsed ones are NULL). Before reading the range, the metadata lock is upgraded to MDL_SHARED_NO_WRITE, and then the copy does not run online, so DML cannot add history that would not fit the computed layout until the ALTER ends (Alter_info::vers_history_read). LOCK=NONE and ALGORITHM=INPLACE/NOCOPY/INSTANT are rejected for such ALTER before the upgrade, so they do not wait for concurrent DML; the ALGORITHM error is reported by the new Alter_info::report_algorithm_unsupported(), which mysql_alter_table() uses as well. The range is read under a table lock taken with mysql_lock_tables(), as some engines (Aria) set up their transaction in THR_LOCK callbacks. The scan can be killed. WARN_VERS_STARTS_BEYOND_INTERVAL for ALTER .. AUTO is checked after the alignment, so it is not printed when STARTS was re-aligned to the history. vers_get_history_range() detects if an index can be used for row_end min/max stats and if so it gets it with ha_index_first() and HA_READ_BEFORE_KEY (as it must ignore current data). Otherwise it does a table scan to read the stats. Current rows with the pre-11.5 row_end marker are not taken for history either. In the index they sort below history after 2038, so the minimum is read after them. There is test_mdev-25529 debug keyword to check the both and compare results. WARN_VERS_SLOW_ROW_END is printed if the scan read any row and there is no usable index (a disabled one does not count as missing). Static key_cmp was renamed to key_eq to resolve compilation after key.h was included as key_cmp was already declared there. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41634 Rename Item::deep_copy() to pushdown_copy() After the previous commits, every caller of Item::deep_copy_with_checks() outside the copy methods themselves copies a condition for pushdown, apart from two debug hooks and Virtual_column_info::clone(), which has no callers. The copies are pushed into a materialized derived table or view, into a materialized IN subquery, or from HAVING into WHERE. The pushdown code transforms the copy, replacing the columns it refers to with what the target select computes them from, then walks it with cleanup_excluding_const_fields_processor and fixes it again in the target select. That walk cleans up every node it reaches, and Item_field::cleanup() drops the field. So no node of the copy that the walk reaches may be shared with the original, which is what deep_copy() already provides. The one shared node is the example of an Item_cache over an aggregate or window function, which deep_copy() could never copy. The walk does not reach it, because every transformer the pushdown code applies before the walk replaces an Item_cache with a basic constant, or fails and the condition is not pushed. main.derived_cond_pushdown (MDEV-10882) copies such a cache. Rename deep_copy(), deep_copy_with_checks() and Item_cache:: deep_copy_cache() to pushdown_copy(), pushdown_copy_with_checks() and Item_cache::pushdown_copy_cache(), and document these requirements at pushdown_copy_with_checks() and Item_cache::pushdown_copy_cache(). The code is otherwise unchanged, as is the behaviour: the existing pushdown tests cover it. The next commit removes the debug hooks and Virtual_column_info::clone(). This commit was prepared with Claude Code (Opus 5), which found that pushdown is the only remaining user of deep_copy(), compared every deep_copy() override with what pushdown needs, checked with tracing that the pushdown cleanup walk never reaches the aggregate that a copy of an Item_cache shares, renamed the hierarchy, rewrote its comments, and ran main and the SQL-layer suites in normal and --ps-protocol mode. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41632 UPPER(key_col) IN (constant subquery) returns incorrect row Item_func_in::varchar_upper_cmp_transformer() rewrites UPPER(key_col) IN (const, ...) into key_col IN (const, ...). It needs a copy of the IN predicate to put the new first argument into, it used deep_copy_with_checks(). A deep copy copies every node of the tree and refuses as a whole if any node cannot be copied, so an IN list holding an uncorrelated scalar subquery, which is constant but has no copy, silently lost the rewrite: UPPER(a) IN ((SELECT MAX(c) FROM t2),'A-2') read the whole table instead of using a range on the index on a. After copying, the rewrite walked the copy with cleanup_excluding_const_fields_processor and re-fixed it, which would also have cleaned up a subquery had one been copied. A copy for use in the same SELECT_LEX does not need to copy what can be shared there. We add a separate copy hierarchy for that purpose. Item::same_select_copy_with_checks() calls the protected virtual Item::same_select_copy(), and cleans up every node it actually copied, so that the caller only has to call fix_fields() on the result. fix_fields() re-fixes the copied nodes and skips the shared ones, which are still fixed. It does not use deep_copy() at any level; the only other copy function it calls is shallow_copy() on the node being copied. Item default is to share the item. Item_func, Item_cond and Item_row copy their node and recurse into their arguments. Item_func copies an argument that occurs more than once in args[] only once. Item_bool_rowready_func2, Item_func_case_simple and Item_func_in detach the comparators and value array they would otherwise share with the original, as their deep_copy() overrides do. Fields, references, constants, parameters, caches, subqueries, aggregate and window functions are shared because they don't derive from Item_func. Some Item_func subclasses cannot safely be copied, so they override same_select_copy() to share the original instead: - Functions that own memory or handles, which the copy would free a second time in cleanup() or its destructor: the regex functions, UDFs, stored functions, the JSON functions that build hashes, VEC_FromText, and the geometry functions with Gcalc state. - Functions that their SELECT_LEX keeps a pointer to, so a copy would not be the item the SELECT_LEX uses: MATCH, ROWNUM and Item_equal. - Functions whose copy would behave differently: RAND would restart its random sequence, and SET @var would assign the variable twice. - TO_NUMBER, which refuses to be copied at all. - Functions whose arguments point at each other: NULLIF uses args[0] a second time as args[2], and Item_in_optimizer's cache belongs to its subquery. The rewrite now uses same_select_copy_with_checks() and no longer walks the copy with a cleanup processor. TABLE::add_splitting_info_for_key_field() uses it too, for the left side of the equality it pushes into a split-materialized derived table: the item of the derived table's select list, used in the WHERE condition of the same select. That item is always a column, so the copy is the column itself, where the deep copy made a second Item_field for it. The equality is fixed when it is injected. The split_materialized_clones check of the left side is removed, as it is no longer meant to be a deep copy. This commit was prepared with Claude Code (Opus 5), which found the subquery case that lost the rewrite and confirmed that the new test fails without the change, designed and wrote the same_select_copy() hierarchy and the list of shared Item_func classes, switched the casefold rewrite to it, and ran main and the SQL-layer suites in normal and --ps-protocol mode. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 ErrConvTimestamp for printing timestamps Add ErrConvTimestamp, an ErrConv that formats a Timestamp in the session time zone into a fixed buffer for warning and error messages, with the number of fractional digits given explicitly. ErrBuff::set_mysql_time() gets an optional digits argument for it. Use it when printing STARTS and the history range. Rename the error symbol ER_PART_STARTS_BEYOND_INTERVAL to WARN_VERS_STARTS_BEYOND_INTERVAL and extend its message: it now takes the STARTS timestamp and the query timestamp as arguments, so the number of format arguments changes from one to three. The error number (4164) is unchanged, so this is not an ABI break, but libmariadb still exports the old symbol name for that number until its submodule is updated. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Nikita Malyavin
nikitamalyavin@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41238 Bound parameters prevent condition pushdown into a grouped view, while CAST enables it Item_param::deep_copy() returned NULL, and every rewrite that copies a condition gives up on what it cannot copy. A conjunct holding a prepared statement parameter was therefore left out of: - pushdown into derived tables and views, including UNION branches and conditions on a derived table column whose expression holds a parameter; - pushdown from HAVING to WHERE, for the part extracted from an OR; - pushdown into a materialized IN subquery; - split materialization, which then found no split key; - the UPPER(col) IN (...) rewrite, which has skipped such conjuncts since MDEV-40006. When the derived table groups by an expression with a parameter, the condition was pushed into its HAVING instead of its WHERE. CAST(? AS ...) worked around this only by accident. Before the pushdown, constant propagation can replace a constant expression on one side of = with a literal. A bare parameter already counts as a basic constant, so propagation leaves it as it is. A client whose charset differs from the column's got the pushdown by a similar accident: the charset conversion replaces the parameter with a literal. Item_param::shallow_copy() now copies a parameter that holds a value, NULL included, and deep_copy() calls it, as in other leaf items. Every such copy is made while optimizing during EXECUTE, and the next EXECUTE makes a new one, so a copy always carries the value bound for the EXECUTE that made it. A copy that outlived its EXECUTE would keep an old value and point into a string buffer that the next binding may free, so shallow_copy() asserts that it does not allocate on the statement's arena. The copy is not added to m_clones. m_clones keeps the parameter's copies in CTE definitions updated with each new value for the whole life of the statement, while this copy lives for one execution. An unbound parameter, DEFAULT and IGNORE still have no copy. mysqltest gets a PS_bind_long command. It binds like PS_bind, but sends every value with mysql_stmt_send_long_data() before each PS_execute, so a test can reach a parameter in LONG_DATA_VALUE state. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40012 PQ: refuse DEFAULT() in an expression a worker evaluates DEFAULT(col) was evaluated against the worker's row. Item_default_value is an Item_field whose field is a private copy made by make_default_field(), reading the share's default_values but carrying the column's own table. Its deep_copy() shares that field, and Pwt_field_rebinder, seeing the manager's table, repointed it at the worker's real column. DEFAULT(col) then read the current row's value, and for an expression default calculate() called set_default() and wrote the default into the row -- wrong results in a release build, a marked_for_write assertion in a debug one. It shows only when the expression also names the column some other way; alone, DEFAULT(col) is table-independent and evaluated at optimize time. The gate now refuses any expression holding a DEFAULT(), found with check_func_default_processor, which only Item_default_value answers. Found by running the main suite with --parallel-worker-threads=4, where type_temporal_innodb lost a "Truncated incorrect BOOLEAN value" warning because the worker's WHERE evaluated to NULL. parallel_query_clone covers a lost row, a gained row and the expression default that crashed, and checks that none of them ran in the workers. This commit was prepared with Claude Code (Opus 5.5), which traced the wrong result from a main-suite sweep with workers enabled to the rebinder, wrote the gate refusal and the test, confirmed the test fails without the refusal, and ran the parallel_query tests in normal and --ps-protocol modes. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40287 AES_ENCRYPT() and KDF() return NULL with OpenSSL 4.0 MyCTX used an EVP_CIPHER_CTX in a stack buffer instead of allocating it with EVP_CIPHER_CTX_new(). OpenSSL 4.0 rejects a context that was not created that way for ciphers that use an IV: EVP_CipherInit_ex() fails with "invalid iv length", so AES_ENCRYPT() and KDF() in CBC/CTR/GCM modes return NULL. ECB has no IV and still works, which is why only the non-ECB modes broke. Allocate the context with EVP_CIPHER_CTX_new()/EVP_CIPHER_CTX_free(), and on an init error free and null it, so a later update()/finish() from a caller that ignored the error is a safe no-op, not a double free. Remove check_openssl_compatibility() and the EVP_CIPHER_CTX_SIZE and EVP_CIPHER_CTX_init macros. my_md5() used the same stack-buffer-sized-by-EVP_MD_CTX_SIZE hack, whose only guard was check_openssl_compatibility(). MD5_CTX is a concrete struct in both OpenSSL and WolfSSL, so use it directly with MD5_Init/Update/Final on a caller-owned context, like my_sha.inl does for SHA*_CTX; this removes the EVP_MD_CTX hack and EVP_MD_CTX_SIZE. The exception is OpenSSL 1.0.x, which aborts on the low-level MD5 API in FIPS mode (MDEV-7788): there, and only there, keep the EVP path with the non-FIPS flag. 1.1.0 removed the FIPS Object Module and 3.0+ low-level MD5 bypasses the provider, so the concrete MD5_CTX is correct on every other version. SHA is unaffected, it already used concrete SHA*_CTX structs. Verified against OpenSSL 4.0.1: the mysys aes-t test fails on the CBC/CTR/GCM cases with the stack buffer and passes with EVP_CIPHER_CTX_new(). The 1.0.x MD5 path verified against a FIPS build of OpenSSL 1.0.2u: the EVP path computes MD5, the low-level path aborts. No visible performance degradation: the extra allocation costs ~12 ns/call (WolfSSL) and ~30 ns (OpenSSL) on Windows at a 30-byte payload, nothing at 16 KB, and nothing on Linux/glibc; sysbench OLTP over encrypted tables and redo log is unchanged. Tests main.tlsv13 and main.ssl_crl: OpenSSL 4.0.3 reports TLS alerts as "tls alert ..." (was "ssl/tls alert ..."), accept both. Assisted-by: Claude:claude-opus-4-8 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40012 PQ: a worker must not end its container's scan after handing it over A pre-aggregating worker accumulates into its own container, a temporary table, and hands that container to the manager with sink->flush(). Just before the hand-off, execute_and_handoff() closes the container's index, because the manager starts its own scan of the container immediately after the hand-off. The tidy-up at exec_exit then closed the container a second time. Its comment said this was only for the paths that reach exec_exit without a hand-off, but the code ran on every path, including the path with a successful hand-off. By that point the container may already belong to the manager: claim_next_result() has seen it marked as done, and pwt_tmp_table_source::next_row() has rebound it to the manager's thread and called ha_rnd_init() on it. The worker's ha_index_or_rnd_end() therefore ends the manager's scan. The manager's next ha_rnd_next() then fails "inited == RND" in a debug build, and in a release build it reads from a table with a closed scan. The window between the flush and the second close is only a few instructions wide, so the failure is rare: main.ps_3innodb hit it once in a main-suite run with --parallel-worker-threads=4, on SELECT ... UNION DISTINCT SELECT sum(a) ... GROUP BY b. The second close now runs only on the paths that skipped the first close, as recorded by a flag set next to the first one. parallel_query_handoff makes the race certain. With pwt_handoff_sync set, a worker that has handed over its container waits until the manager reaches the new sync point pwt_manager_scanning_container, and the manager keeps its scan open until the worker has finished its tidy-up. The test uses one worker, so there is a single container and no risk of crossed signals. It checks the rows and that the query ran in the worker. Without the fix it fails the inited == RND assertion on every run. This commit was prepared with Claude Code (Opus 5.5), which found the crash in a main-suite sweep with workers enabled, traced it from the core dump to the second close, wrote the fix, the sync points and the test, and confirmed that the test fails without the fix in five runs out of five and passes with it in normal and --ps-protocol modes. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 ALTER TABLE FORCE syntax improved Lets an alter_list be supplied alongside a partitioning or remove_partitioning clause in either order. This allows the FORCE clause to be added to an existing command, and lets AUTO and FORCE be specified together, which later commits of this task rely on. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41508 Off-by-One Uninitialized Stack Read in Proxy Protocol V2 Port Parsing parse_v2_header() required only 25 (TCPv4) / 49 (TCPv6) bytes, but then reads the source port at hdr[25] / hdr[49]. A truncated header could make it read uninitialized stack. Require the whole address block: 28 bytes for TCPv4 and 52 for TCPv6. Also, the declared header length was trusted without checking how many bytes vio_read() returned. Reject short reads. The spec requires the sender to send the header at once, and allows the receiver to drop the connection on a partial header, so a single vio_read() is enough. The v1 parser had the same problem on EOF: vio_read() returning 0 was taken for a byte read, so the uninitialized buffer was parsed. Require exactly one byte. Extend test_proxy_header_tcp with a header cut short before the port, and add test_proxy_header_short for a header shorter than it declares. Light refactoring, to avoid repetition: added connect_tcp() and init_v2_header(), which are now shared by several proxy header tests. Assisted-by: Claude:claude-sonnet-5-5 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41251 Wrong result with split materialized query A query like SELECT * FROM t0 WHERE t0.b IN (SELECT t.n2 FROM t1, (SELECT n1, n2 FROM t1 GROUP BY n1, n2) t WHERE t.n1 = t1.n1 AND t1.c1 = 'a' UNION SELECT -1) returned only the row of t0 that was read first, when the derived table t was split-materialized. The IN->EXISTS transformation wraps t0.b in an Item_cache owned by the Item_in_optimizer, and injects t.n2 = <cache>(t0.b) into the subquery. The optimizer then pushes an equality on n2 into the derived table, so that the table only has to be filled with the matching rows. TABLE::add_splitting_info_for_key_field() built the right side of that equality as a deep copy of <cache>(t0.b). A copy of a cache is a second cache: it read t0.b for the first row of t0 and kept that value, because only the Item_in_optimizer stores new values, and it stores them into the original cache. Every later refill of the derived table was filtered on the first row's value. The right side is an expression of the outer select used inside the select that specifies the derived table. Add a copy hierarchy for that purpose. Item::new_select_copy_with_checks(thd, item_select) returns a copy for use in a select nested in item_select, the select the item belongs to. In the nested select the copy is an outer reference to item_select: - An Item_field is copied and its depended_from set to item_select. This is what add_splitting_info_for_key_field() did before with set_fields_as_dependent_processor, except that depended_from now names the select the column belongs to instead of the nested select. - An Item_direct_view_ref, a column of a merged view or derived table, is also copied like a column: the copy refers to a new_select_copy() of the item the view column stands for. Copying it this way, as the deep copy did, keeps a column of a constant NULL-complemented table constant in the pushed equality (the MDEV-30081 test in main.derived_cond_pushdown still shows ref "const" for its LATERAL DERIVED table). - Any other item that depends on tables of item_select is not copied. The copy is an Item_direct_ref to the original item, with depended_from set to item_select. So the nested select reads the cache, aggregate or subquery that its owner keeps up to date, instead of a copy of it that nobody refreshes. Item_default_value is treated this way too. - An item that depends on no table of item_select is shared. The copy is fixed, and only refers to tables outside the select it is used in, which the wrapper asserts. add_splitting_info_for_key_field() now uses it for the right side of the pushed equality, and no longer walks the copy with set_fields_as_dependent_processor or calls update_used_tables() on it. The debug-only copy of key_field->val used for the lookup still uses deep_copy_with_checks(). The check_deep_copy() call on the right side is removed, as it is no longer meant to be a deep copy. A second test, taken from Dave Gosselin's MDEV-41251 commit, pushes a comparison, dt.a <=> (t1.b = 1), and its row form. The deep copy of the comparison was never fixed again. Its Arg_comparator still pointed at the original, which sets null_value of the original, and a copied row comparison had comparators set to NULL. The row of t1 with a NULL b was missing from the result, and the row form crashed in Arg_comparator::compare_row(). Now the comparison is not copied: the right side refers to the original. This commit was prepared with Claude Code (Opus 5), which designed and wrote the new_select_copy() hierarchy, switched the split equality to it, wrote the three-row test from the earlier rr analysis, added the comparison test, confirmed in a build of the parent commit that both tests fail without the change, and ran main and the SQL-layer suites in normal and --ps-protocol mode. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41631 Check the class of the copy made by Item::shallow_copy() Item::shallow_copy() was meant to create a copy of the same class as the item it is called on, but nothing enforced that. A class that does not declare its own override inherits the one of its nearest concrete parent, and get_item_copy<Parent>() then silently creates an object of the parent class, losing the child's virtual behaviour (see MDEV-38801 for an earlier instance with Item_cache_year). Add the ITEM_SHALLOW_COPY(CLASS) macro, which declares the override, and Item::dbug_check_shallow_copy_type(), a debug-only check called from Query_arena::free_items() for every item freed. Once per dynamic type it creates a shallow copy, unlinks it from thd->free_list again (the copy shares pointers with the original and must not be destroyed) and asserts that the copy has the same typeid as the original. The copy is never destroyed: its destructor is not run, and its memory is released when the MEM_ROOT it was allocated on, the current thd->mem_root, is freed. A comment in the check says so. The check found these classes copied as their parent class, now fixed with ITEM_SHALLOW_COPY(): Item_join_operator_plus, Item_null_result, Item_old_field, Item_datetime, Item_empty_string, Item_return_int, Item_trigger_type_of_statement, in_string::Item_string_for_in_vector, Item_is_not_null_test, Item_interval_DDhhmmssff_typecast, Item_string_xml_non_const, Item_func_trt_trx_sees_eq, Item_field_refcursor (type_cursor plugin), Item_field_packable_scalar (type_assoc_array plugin). Item_field_packable_row::shallow_copy() asserted to mark the class as never copied, which the check trips over since it copies every type it meets. It now returns nullptr, the existing way for an item to refuse a copy. No test is added: the check runs under every existing test in a debug build. main and the SQL-layer suites (compat/*, json, funcs_*, vcol, gcol, versioning, period, sys_vars, type_*) pass with it in both normal and --ps-protocol mode. This commit was prepared with Claude Code (Opus 5), which wrote the runtime type check, ran the main and SQL-layer suites in normal and --ps-protocol mode to find the classes that inherited a parent's copy method, added the ITEM_SHALLOW_COPY overrides for them, changed Item_field_packable_row to return nullptr, and documented where the copy made by the check is released. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Nikita Malyavin
nikitamalyavin@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41238 Bound parameters prevent condition pushdown into a grouped view, while CAST enables it Item_param::deep_copy() returned NULL, and every rewrite that copies a condition gives up on what it cannot copy. A conjunct holding a prepared statement parameter was therefore left out of: - pushdown into derived tables and views, including UNION branches and conditions on a derived table column whose expression holds a parameter; - pushdown from HAVING to WHERE, for the part extracted from an OR; - pushdown into a materialized IN subquery; - split materialization, which then found no split key; - the UPPER(col) IN (...) rewrite, which has skipped such conjuncts since MDEV-40006. When the derived table groups by an expression with a parameter, the condition was pushed into its HAVING instead of its WHERE. CAST(? AS ...) worked around this only by accident. Before the pushdown, constant propagation can replace a constant expression on one side of = with a literal. A bare parameter already counts as a basic constant, so propagation leaves it as it is. A client whose charset differs from the column's got the pushdown by a similar accident: the charset conversion replaces the parameter with a literal. Item_param::shallow_copy() now copies a parameter that holds a value, NULL included, and deep_copy() calls it, as in other leaf items. Every such copy is made while optimizing during EXECUTE, and the next EXECUTE makes a new one, so a copy always carries the value bound for the EXECUTE that made it. A copy that outlived its EXECUTE would keep an old value and point into a string buffer that the next binding may free, so shallow_copy() asserts that it does not allocate on the statement's arena. The copy is not added to m_clones. m_clones keeps the parameter's copies in CTE definitions updated with each new value for the whole life of the statement, while this copy lives for one execution. An unbound parameter, DEFAULT and IGNORE still have no copy. mysqltest gets a PS_bind_long command. It binds like PS_bind, but sends every value with mysql_stmt_send_long_data() before each PS_execute, so a test can reach a parameter in LONG_DATA_VALUE state. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40287 Restore the MDEV-7788 FIPS fix for OpenSSL 1.0.x only OpenSSL 1.0.x in FIPS mode aborts on the low-level MD5 API - the only version that ever needed the EVP detour. 1.1.0 removed the FIPS Object Module and 3.0+ low-level MD5 bypasses the provider, so the concrete MD5_CTX of the previous commit is correct everywhere else. Keep the EVP/non-FIPS path for OpenSSL 1.0.x only (e.g. RHEL 7 ELS) so my_md5() does not abort mysqld under FIPS. A separate commit so it can be dropped once 1.0.x is gone. Assisted-by: Claude:claude-opus-4-8 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sergei Golubchik
serg@mariadb.org |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MDEV-41630 ST_IsValid crashes on polygons with many rings | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41351 Allow a covering index scan for a SELECT ... FOR UPDATE under a full-scan table lock MDEV-24813 introduced a switch innodb_table_lock_on_full_scan that requests a table lock on an innodb table for some unconditional SELECT/UPDATE/DELETE statements. With a full table lock, and when doing a covering scan, there is no need to request clustered index reads. This patch does so for X-locks in SELECT statements. Prior to this it is already the case for S-locks. There is one exception: we disable this optimization when inside a transaction with innodb_snapshot_isolation and a read view already open, in which case it needs trx_id obtained from the clustered index to check if there is a committed change the transaction never saw. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yuchen Pei
ycp@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41351 Allow a covering index scan for a SELECT ... FOR UPDATE under a full-scan table lock MDEV-24813 introduced a switch innodb_table_lock_on_full_scan that requests a table lock on an innodb table for some unconditional SELECT/UPDATE/DELETE statements. With a full table lock, and when doing a covering scan, there is no need to request clustered index reads. This patch does so for X-locks in SELECT statements. Prior to this it is already the case for S-locks. There is one exception: we disable this optimization when inside a transaction with innodb_snapshot_isolation and a read view already open, in which case it needs trx_id obtained from the clustered index to check if there is a committed change the transaction never saw. Co-Authored-By: Claude Opus 5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40287 Use MD5_CTX directly for MD5, like the SHA wrappers my_md5() faked an EVP_MD_CTX in a stack buffer sized by EVP_MD_CTX_SIZE, the same opaque-context hack MyCTX used for the cipher - and its only guard, check_openssl_compatibility(), was removed earlier in this patch. MD5_CTX is a concrete struct in both OpenSSL and WolfSSL (WolfSSL provides the OpenSSL-compatible MD5 API), so use it directly with MD5_Init/Update/Final on a caller-owned context, exactly like my_sha.inl does for SHA*_CTX. This removes the EVP_MD_CTX hack and EVP_MD_CTX_SIZE; my_md5_context_size() now returns sizeof(MD5_CTX). SHA is unaffected - it already used concrete SHA*_CTX structs. Assisted-by: Claude:claude-opus-4-8 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41621 Stored generated BLOB column is wrong on REPLACE into versioned table REPLACE into a system-versioned table turns the conflicting row into history in TABLE::delete_row(): it saves the new row into record[2], restores the conflicting row from record[1] and sets its row_end with vers_update_end(). The VCOL_UPDATE_FOR_WRITE computation in vers_update_end() also recomputes stored generated columns. When Field_blob::store() copies the value into Field_blob::value (e.g. on a character set conversion), it overwrote the buffer that the stored generated blob of the new row in record[2] points to, and the new row was written with the value of the history row. The index on the column holds the same value, so CHECK TABLE does not detect it. remember_blob_values(), which keeps Field_blob::value around vers_update_end() in TABLE::delete_row() and vers_insert_history_row() and around update_generated_fields() in period_make_insert(), covered only virtual columns. The fix adds the with_stored parameter to remember_blob_values(), restore_blob_values() and free_blob_values(), which also covers stored generated blob columns. TABLE::delete_row(), vers_insert_history_row() and period_make_insert() use it and size the storage by TABLE_SHARE::virtual_fields, the number of all generated columns. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Raghunandan Bhat
raghunandan.bhat96@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-34817: Performance schema is not cleared of removed package routines - part 2 Problem: 1. A package routine and a standalone routine can have the same name, e.g. routine p1 of package pkg1 and procedure `pkg1.p1`. Performance schema stored both under one key, so their statistics were merged, and DROP PACKAGE BODY pkg1 also removed the statistics of standalone routines named `pkg1.<anything>`. 2. Performance schema truncates names to COL_OBJECT_NAME_SIZE bytes. For a package name of that length or longer, "<package>.<routine>" became just "<package>", and these records were never dropped. Solution: Give package routines their own object types, OBJECT_TYPE_PACKAGE_PROCEDURE and OBJECT_TYPE_PACKAGE_FUNCTION. They are used only in the hash key, so SETUP_OBJECTS and OBJECT_TYPE columns still see PROCEDURE and FUNCTION. When a package body is dropped, only records with these types are dropped, including names truncated to the package name. Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Nikita Malyavin
nikitamalyavin@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-41238 Bound parameters prevent condition pushdown into a grouped view, while CAST enables it Item_param::deep_copy() returned NULL, and every rewrite that copies a condition gives up on what it cannot copy. A conjunct holding a prepared statement parameter was therefore left out of: - pushdown into derived tables and views, including UNION branches and conditions on a derived table column whose expression holds a parameter; - pushdown from HAVING to WHERE, for the part extracted from an OR; - pushdown into a materialized IN subquery; - split materialization, which then found no split key; - the UPPER(col) IN (...) rewrite, which has skipped such conjuncts since MDEV-40006. When the derived table groups by an expression with a parameter, the condition was pushed into its HAVING instead of its WHERE. CAST(? AS ...) worked around this only by accident. Before the pushdown, constant propagation can replace a constant expression on one side of = with a literal. A bare parameter already counts as a basic constant, so propagation leaves it as it is. A client whose charset differs from the column's got the pushdown by a similar accident: the charset conversion replaces the parameter with a literal. Item_param::shallow_copy() now copies a parameter that holds a value, NULL included, and deep_copy() calls it, as in other leaf items. Every such copy is made while optimizing during EXECUTE, and the next EXECUTE makes a new one, so a copy always carries the value bound for the EXECUTE that made it. A copy that outlived its EXECUTE would keep an old value and point into a string buffer that the next binding may free, so shallow_copy() asserts that it does not allocate on the statement's arena. The copy is not added to m_clones. m_clones keeps the parameter's copies in CTE definitions updated with each new value for the whole life of the statement, while this copy lives for one execution. An unbound parameter, DEFAULT and IGNORE still have no copy. mysqltest gets a PS_bind_long command. It binds like PS_bind, but sends every value with mysql_stmt_send_long_data() before each PS_execute, so a test can reach a parameter in LONG_DATA_VALUE state. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
PQ: Code tidy ups and fixups of tidyups. parallel_query_aggregate.test: fix statistics impersonalize language in can_parallel_scan_jointab_access() remove dead code (trace_parallel_scan_options) add an assert to ensure we do not use Item_sum_avg without two packed fields minor comment fixups remove enum element NESTED_LOOP_DECLINED and use a local bool instead fix silly code in pwt_row_layout::build_aggregates (throwing away a pointer then searching for it again later); 'one' starts as nullptr, because GCC cannot see that whole_table is unchanged between the two blocks and warns -Wmaybe-uninitialized in an ASAN build remove pwt_batch* worker->manager streaming functions tidy up Parallel_scan_partitioner::Scan_ctx::create_chunks to use create_chunks_unsplit The 'one= nullptr' initialisation was added with Claude Code (Opus 5.5), which traced the -Wmaybe-uninitialized warning to this change and folded the fix in here. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Vladislav Vaintroub
vvaintroub@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40287 AES_ENCRYPT() and KDF() return NULL with OpenSSL 4.0 MyCTX used an EVP_CIPHER_CTX in a stack buffer instead of allocating it with EVP_CIPHER_CTX_new(). OpenSSL 4.0 rejects a context that was not created that way for ciphers that use an IV: EVP_CipherInit_ex() fails with "invalid iv length", so AES_ENCRYPT() and KDF() in CBC/CTR/GCM modes return NULL. ECB has no IV and still works, which is why only the non-ECB modes broke. Allocate the context with EVP_CIPHER_CTX_new()/EVP_CIPHER_CTX_free(), and on an init error free and null it, so a later update()/finish() from a caller that ignored the error is a safe no-op, not a double free. Remove check_openssl_compatibility() and the EVP_CIPHER_CTX_SIZE and EVP_CIPHER_CTX_init macros. my_md5() used the same stack-buffer-sized-by-EVP_MD_CTX_SIZE hack, whose only guard was check_openssl_compatibility(). MD5_CTX is a concrete struct in both OpenSSL and WolfSSL, so use it directly with MD5_Init/Update/Final on a caller-owned context, like my_sha.inl does for SHA*_CTX; this removes the EVP_MD_CTX hack and EVP_MD_CTX_SIZE. The exception is OpenSSL 1.0.x, which aborts on the low-level MD5 API in FIPS mode (MDEV-7788): there, and only there, keep the EVP path with the non-FIPS flag. 1.1.0 removed the FIPS Object Module and 3.0+ low-level MD5 bypasses the provider, so the concrete MD5_CTX is correct on every other version. SHA is unaffected, it already used concrete SHA*_CTX structs. Verified against OpenSSL 4.0.1: the mysys aes-t test fails on the CBC/CTR/GCM cases with the stack buffer and passes with EVP_CIPHER_CTX_new(). The 1.0.x MD5 path verified against a FIPS build of OpenSSL 1.0.2u: the EVP path computes MD5, the low-level path aborts. No visible performance degradation: the extra allocation costs ~12 ns/call (WolfSSL) and ~30 ns (OpenSSL) on Windows at a 30-byte payload, nothing at 16 KB, and nothing on Linux/glibc; sysbench OLTP over encrypted tables and redo log is unchanged. Tests main.tlsv13 and main.ssl_crl: OpenSSL 4.0.3 reports TLS alerts as "tls alert ..." (was "ssl/tls alert ..."), accept both. Assisted-by: Claude:claude-opus-4-8 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| get_next_time() comment | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Aleksey Midenkov
midenok@gmail.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-25529 Auto-create: Pre-existing historical data is not partitioned as specified by ALTER Adds logic into prep_alter_part_table() for INTERVAL ... AUTO to read the history range (vers_get_history_range()) and size the new layout from it. STARTS is aligned to the oldest history row_end (vers_set_starts()) if it was not specified by the user, or if the user specified it later than that; in the latter case WARN_VERS_WRONG_STARTS is printed. The history partitions cover both the newest history row_end and the query time, so DML after the ALTER does not need to create partitions. A PARTITIONS N clause is kept if it gives more partitions. An explicit subpartition list is replaced by as many default subpartitions. If the layout needs more than MAX_PARTITIONS partitions (counting subpartitions), or the INTERVAL is longer than the date range, the ALTER fails with ER_VERS_HIST_PART_FAILED. Month and year intervals are stepped in UTC; a boundary past the maximum TIMESTAMP ends a partial last history partition. In case of fast ALTER TABLE, f.ex. when partitioning already exists, the above logic is ignored unless FORCE clause is specified. When user specifies partition list explicitly the above logic is ignored even with FORCE clause. With FORCE on a table already partitioned by SYSTEM_TIME the work is done on a clone of the partitioning info. The parsed settings are copied into the clone's vers_info with Vers_part_info::copy_settings(), which keeps the clone's now_part and hist_part (the parsed ones are NULL). Before reading the range, the metadata lock is upgraded to MDL_SHARED_NO_WRITE, and then the copy does not run online, so DML cannot add history that would not fit the computed layout until the ALTER ends (Alter_info::vers_history_read). LOCK=NONE and ALGORITHM=INPLACE/NOCOPY/INSTANT are rejected for such ALTER before the upgrade, so they do not wait for concurrent DML; the ALGORITHM error is reported by the new Alter_info::report_algorithm_unsupported(), which mysql_alter_table() uses as well. The range is read under a table lock taken with mysql_lock_tables(), as some engines (Aria) set up their transaction in THR_LOCK callbacks. The scan can be killed. WARN_VERS_STARTS_BEYOND_INTERVAL for ALTER .. AUTO is checked after the alignment, so it is not printed when STARTS was re-aligned to the history. vers_get_history_range() detects if an index can be used for row_end min/max stats and if so it gets it with ha_index_first() and HA_READ_BEFORE_KEY (as it must ignore current data). Otherwise it does a table scan to read the stats. Current rows with the pre-11.5 row_end marker are not taken for history either. In the index they sort below history after 2038, so the minimum is read after them. There is test_mdev-25529 debug keyword to check the both and compare results. WARN_VERS_SLOW_ROW_END is printed if the scan read any row and there is no usable index (a disabled one does not count as missing). Static key_cmp was renamed to key_eq to resolve compilation after key.h was included as key_cmp was already declared there. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Rex Johnston
rex.johnston@mariadb.com |
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MDEV-40012 PQ: end a worker's parallel scan before close() frees m_prebuilt A worker's handler was destroyed with its parallel scan still open. exec() and scan_only() jump to their exit labels when parallel_init_worker() fails, and the commonest failure is HA_ERR_END_OF_FILE: no chunk was left for this worker, which happens all the time on small tables. InnoDB has allocated m_pscan_worker by then, so the scan was left for ~ha_innobase() -> parallel_scan_free(), which runs after close() has freed m_prebuilt. parallel_scan_free() says it drops the worker without end() for exactly that reason, but ~Parallel_scan_worker() called end() anyway, and end() writes m_prebuilt->pscan_chunk_clamp: a heap-use-after-free that ASAN reports on every such worker. In a build without ASAN the write lands in freed memory, and it is the likely cause of the intermittent SIGSEGV in free() inside pthread_create() when a later worker reused a cached thread stack whose DTV had been overwritten. Three changes close it. Both workers now end the scan at their exit label instead of after the loop, so every way out ends it while the handler is still open; ending a scan that was never begun is harmless. ha_innobase::close() ends any worker or coordinator scan still open before row_prebuilt_free(), so a path that misses it is ended properly rather than left for the destructor; with the SQL layer put back as it was, this alone keeps ASAN clean. And ~Parallel_scan_worker() no longer calls end(), so parallel_scan_free(), now reached only for a handler destroyed without close(), cannot write through m_prebuilt either. The close() hook and the single call site are taken from a patch by a co-worker. Two debug assertions state the rule the defect broke, so it no longer takes ASAN to see it. ha_innobase::close() asserts that the SQL layer has already ended a worker's scan, and parallel_scan_free() asserts that no worker survives close(). The injection pwt_leave_scan_for_close makes the SQL layer leave the scan of a worker that holds no cursor -- one handed no chunk -- for close() to end, so the backstop is exercised on purpose and the first assertion is told to allow it. pwt_end_worker_scan() holds the injection and ends the scan at both exit labels. parallel_query_no_chunk (debug only) runs a full scan and a range scan of a one-page table with four workers, so three of them get no chunk, twice each, through the normal and the scan-only path, with and without the injection, checking the rows and that the workers ran every query. In a plain debug build, removing the exit-label calls fails the first assertion and removing the close() backstop fails the second. ~Parallel_scan_worker() no longer calling end() is defence for release builds and is not reachable while the backstop is there. This commit was prepared with Claude Code (Opus 5.5), which reproduced the pthread_create() crash under ASAN and traced it, wrote the fix, folded in the co-worker's close() hook and single call site, added the debug assertions, the injection and the test, confirmed in a plain debug build that the test fails with each of the two fixes removed, and ran the parallel_query tests in normal and --ps-protocol modes and the main suite with --parallel-worker-threads=4 under ASAN. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||