Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
MDEV-30555 The server does not detect changes in NULL-ability of system table columns

Table_check_intact::check() validated column names, types, and charsets but
not NULL-ability. Altering a system table column (e.g. mysql.proc.definer)
to drop NOT NULL would pass the check undetected, silently corrupting data
stored through that column.

Add a CAN_BE_NULL flag bit in the high bit of TABLE_FIELD_TYPE::type.length.
Set it on nullable column entries; the check now validates nullability in both
directions. NOT NULL columns need no annotation (the common case). The flag
is encoded as `{ STRING_WITH_LEN("type") + CAN_BE_NULL }` which keeps the
intent readable at the definition site.

Apply CAN_BE_NULL annotations to all TABLE_FIELD_TYPE arrays:
mysql.proc, mysql.event, mysql.table_stats, mysql.column_stats,
mysql.index_stats.

Assisted-By: Claude:claude-4.6-sonnet
Daniel Black
MDEV-40165: json_normalize (fix)

14c16e02b261ad8e41ea6f80ebd4f42403793aff changed the
interface for json_normalize to take a json_engine_t so that
there could be some error checking.

Unfortunately ColumnStore depended on this interface.

Restored original interface of json_normalize() and
added a json_normalize_engine() that take as engine as the
argument.
Alexey (Holyfoot) Botchkov
MDEV-40357 Server crashes in XMLSchema_group_reference::validate_prepare.

The <group> in the XML Schema can't be empty. Launch an error if that
happens.
Sergei Golubchik
MDEV-23486 RBR can bypass secure_timestamp=YES

fix TIMESTAMP DEFAULT NOW() and ON UPDATE NOW()

Caveat: rbr cannot work without an index - if the slave overwrites
timestamp columns, before-image won't match in full, this is expected.
Thirunarayanan Balathandayuthapani
MDEV-39468 Mariabackup: redo log copier stalls and fails with misleading error message

Problem:
========
During backup process, log copier thread stalls at lsn and fails with
the misleading "Was only able to copy log from X to Y, not Z; try increasing
innodb_log_file_size", even when the redo was intact and the log far
from full.

Problem is that copier advances recv_sys.lsn only on CRC-validated mtr,
but it parses the redo at the server is concurrently writing. InnoDB
rewrites the current partial write block repeatedly as mini-transaction
fills it, so an mmap read (or) pread of a page being written can observe
a torn/intermediate image of the tail block. If that tail block happens
to parse as a longer, still valid crc valid mtr then recv_sys.lsn,
recv_sys.offset gets advanced wrongly and points to middle of the
mini-transaction. In that case, later parse return GOT_EOF always
and copier thread never reaches target and fails

Solution:
=========
Poll the server's durably-flushed LSN (status var Innodb_lsn_flushed) and
limit the redo log copier based on it.

backup_log_parse(): parses one mtr and if it exceeds the limit
rolls back recv_sys.lsn/offset and reports GOT_EOF; the copier
re-parses those bytes on a later pass once the fence has
advanced past them. Used on both the mmap and buffered paths.

log_copying_thread(): opens its own connection (it runs
concurrently with the main thread, which owns mysql_connection) and refreshes
the max_limit for parsing of redo log in each pass.

In the final backup phase (BACKUP STAGE BLOCK_COMMIT) it gates
on the exact target max(metadata_last_lsn, metadata_to_lsn).
so the last partial block is copied in full up to the target.

The stall diagnostic / retry spin is suppressed on a reached_parse_limit wait
so a normal "caught up, wait" is not misreported as drift.

get_log_flushed_lsn(): Added to get log_flushed_lsn from SHOW STATUS outout;
Abhishek Bansal
MDEV-38033: JSON_SCHEMA_VALID returns wrong result for array of objects

(11.4 backport from 12.3)

Fix a bug in Json_schema_items::validate where it was missing a
json_skip_level() call. Without this skip, the validator would
incorrectly recurse into non-scalar array elements (like objects)
and try to validate their internal keys/values against the array's
item schema.
Sergei Golubchik
fixup! MDEV-39363
Yuchen Pei
MDEV-31182 [wip] Drop column stats on incompatible changes

ALTER TABLE ... MODIFY ... should drop column stats when the changes
are incompatible for the purpose of preserving the stats. Previously
this was only done when the alter algorithm is not COPY, and the check
is can_convert_no_copy, which checks compatibility for inplace change
rather than column stats. So we fix that by checking
type_handler_for_comparison equality which seems to suggest stats
compatibility (?).

TODO:
- check whether type_handler_for_comparison equality is sufficient
- check whether affected index stats need to be dropped too
Alessandro Vetere
MDEV-40129 Retry transient trylock failures in lock-release fast paths

The trylock attempts on per-cell lock_sys_t::hash_latch (try_acquire())
and on per-table dict_table_t::lock_mutex_trylock() inside
lock_release_try(), lock_release_on_prepare_try() and
lock_rec_unlock_unmodified() now use a bounded spin loop
(up to LOCK_RELEASE_TRY_SPIN_BUDGET CAS attempts, with MY_RELAX_CPU()
between them) instead of a single CAS attempt.

These paths hold trx->mutex while attempting the trylock, which is the
reverse of the standard order used by lock_rec_convert_impl_to_expl().
Blocking acquisition is therefore unsafe, hence the trylock pattern.
However, a single failed CAS marks the entire pass of lock_release_try()
as unsuccessful, and after 5 such failed passes lock_release() falls
back to exclusive lock_sys.wr_lock() for the whole transaction. That
global wr_lock then blocks every concurrent lock_sys.rd_lock() acquirer
in lock_rec_lock() and lock_table(), producing a server-wide convoy
under heavy concurrency.

The bounded spin (no syscall, no blocking) gives a transient latch
holder time to release without weakening the deadlock-avoidance
guarantee that motivated the trylock pattern. The extra trx->mutex hold
time is bounded by LOCK_RELEASE_TRY_SPIN_BUDGET times the pause cost.

This is a first, still to be fine-tuned implementation. Only the
lock_release_try() path has been positively tested; the
lock_release_on_prepare_try() path is not yet covered.
Daniel Black
MDEV-28404 json killed_ptr cleanup

Since 24ee5fd6bf6065ea693287e5705f3e26bb85f9fc the result of the killed_ptr
is part of the je->s.error after json_scan_next.
Daniel Black
MDEV-39742 11.4 JSON functions not interuptable

The JSON functions added in 11.4 where not interuptable
with a KILL QUERY or exceeding the max_statement_time.

* JSON_ARRAY_INTERSECT
* JSON_OBJECT_FILTER_KEYS
* JSON_OBJECT_TO_ARRAY
* JSON_SCHEMA_VALID

This behaviour is corrected by setting the killed_ptr
of the json_engine_t structure.

JSON_KEY_VALUE was added in 11.4, however as it doesn't call
json_next_value(), its processing doesn't check the killed pointer
of the json engine. So its quick anyway.

As a result of added error handling, some warnings are generated.
Daniel Black
MDEV-28404 json killed_ptr cleanup

Since 24ee5fd6bf6065ea693287e5705f3e26bb85f9fc the result of the killed_ptr
is part of the je->s.error after json_scan_next.
Daniel Black
merge - json_normalize changes
Sergei Golubchik
MDEV-28743 Roles without grants are handled wrong

In propagate_role_grants_action() (called by grant_reload() on startup
and FLUSH PRIVILEGES), role privilege merging uses a counter-based
bottom-up ordering: each role's counter tracks how many of its granted
sub-roles still need to be processed, and merge_role_privileges() merges
a role only once its counter reaches zero.

The bug: when an intermediate role such as org_role_1 (which inherits
only from a USAGE-only app_role_1) was merged with no privilege changes,
merge_role_privileges() returned 1 as an optimisation to stop upward
traversal.  This prevented the traversal from ever reaching user_role_1
to decrement its counter.  A second leaf traversal (from app_role_2)
could decrement user_role_1's counter only once, leaving it at 1
instead of 0, so user_role_1 was never merged and had no effective
privileges.

The "stop if nothing changed" optimisation is valid for incremental
propagate_role_grants() calls (after a single GRANT/REVOKE), where
every ancestor was already correctly merged.  It is not valid for the
full-reload case, where the counters must be decremented by every leaf
traversal to guarantee correct bottom-up ordering.

Fix: add an initial_load flag to PRIVS_TO_MERGE.  When set (only in
propagate_role_grants_action), the early-stop optimisation is disabled
so traversals always visit all ancestors.

Assisted-By: Claude:claude-4.6-sonnet
Aleksey Midenkov
MDEV-17613 Read sole now-partition for non-historical versioned query

When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in the
current partition. Restrict read_partitions to that partition in
vers_setup_conds() instead of relying on a row_end condition.

read_partitions is reset to all partitions every statement in
open_table(), so this restriction must be re-applied on each execution.
Guard it with !vers_conditions.was_set() rather than !is_set():
set_all() sets type (flips is_set()) but not orig_type (was_set()), so
is_set() would stay true across prepared-statement / stored-procedure
re-executions and skip the restriction, leaving history partitions
readable - rows matched then grow on each execute/call.

EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.
ParadoxV5
MDEV-39788: Remove added line in `master.info` format

The line-count lines in `master.info` and `relay-log.info`
has been inconsistent (off by one) since their introductions.
MDEV-37530 “fixed” this with its common code merger by chance,
changing `master.info` to use `relay-log.info`’s line-count definition.
This change, howëver, affected backward compatibility,
as `master.info` now expects an ignored MySQL-only line
where the first `key=value` option, `master_use_gtid`, is.

Since this legacy text-based format has limitations that make
it due for replacement, only code reüsablility is valuable,
and its consistency does not outweigh its compatibility.
Therefore, this commit solves this problem without reverting code by
* Changing the writing code to be compatible with both interpretations
  (albeit inconsistent with the reading code)
* Adding a shim entry to `master.info`’s list
  to emulate prior versions’ reading behaviour
  * Although this solution can only restore upgrade compatibility with
    versions 10.0+, versions before MariaDB 10 have long been EOL.

While here, this commit also fixes code and
comments that contradict the actual effect.

Reviewed-by: Brandon Nesterenko <[email protected]>
Georgi (Joro) Kodinov
MDEV-40391: a CMP0218 warning with cmake 4.4

IF() still evaluates the RHS of an AND condition and this causes
the warning here to be thrown for accsseing deprecated variables,
CMAKE_WARN_DEPRECATED in this case.

Fixed by splitting the IF() into two nested IF()s.
Sergei Golubchik
MDEV-24598 Duplicate CHECK constraint names are allowed

Column-level CHECK constraints derive their effective name from the column's
field name when read from frm (name_length=0 fallback in
init_from_binary_frm_image). An explicit table-level constraint with the same
name as a column that carries a column-level check creates a duplicate in
information_schema.check_constraints.

Two cases:
1. CREATE TABLE t (a INT CHECK (a>2), CONSTRAINT a CHECK (a<5)) -- both
  end up named "a".
2. JSON columns auto-receive a JSON_VALID column check named after the field.
  CREATE TABLE t (json_c1 JSON, CONSTRAINT json_c1 CHECK (...)) produces
  a duplicate "json_c1".

Fix in mysql_prepare_create_table_finalize: when validating explicit
table-level constraint names, also check them against field names of columns
that carry column-level check constraints (ER_DUP_CONSTRAINT_NAME).

Fix in make_unique_constraint_name / fix_constraints_names: pass the column
list so that auto-generated names (CONSTRAINT_1, CONSTRAINT_2, ...) skip
names already taken by column-level checks.

Update check_constraint.test: the MDEV-16630 test relied on the now-rejected
duplicate (column check "b" + table constraint "b"). Rename the table-level
constraint to "b_and_a" to preserve the intent: verifying that field
constraints and table constraints produce distinct error-message formats.

Assisted-By: Claude:claude-4.6-sonnet
Daniel Black
MDEV-28404 json killed_ptr cleanup

Since 24ee5fd6bf6065ea693287e5705f3e26bb85f9fc the result of the killed_ptr
is part of the je->s.error after json_scan_next.
Daniel Black
MDEV-40165: json_normalize (fix)

14c16e02b261ad8e41ea6f80ebd4f42403793aff changed the
interface for json_normalize to take a json_engine_t so that
there could be some error checking.

Unfortunately ColumnStore depended on this interface.

Restored original interface of json_normalize() and
added a json_normalize_engine() that take as engine as the
argument.
Rex Johnston
MDEV-17846 Wrong result with grouping select from merged derived table

When a grouping select referring to a derived table has an outer
reference within it's select list, that outer reference isn't
properly fixed.  It isn't added to
outer_context->select_lex->inner_refs_list for later resolution during
JOIN::prepare.  As it is not fixed, the Item pointer arrays associated
with the temporary table filled during the join process are now incorrect
and we end up with an incorrect join->outer_ref_cond evaluation and
potentially a wrong result.
Sergei Golubchik
MDEV-40337 store user vars in a dedicated memroot

user_var_entry objects and their names have a connection lifetime,
they exist until the connection ends (or is reset), and then they're
all deleted at once. This is exactly the use case for MEM_ROOT,
let's store them there.

Additionally, let's set MY_ROOT_USE_MPROTECT flag to keep this memroot
off the general heap where user_var_entry values are stored
and where heap buffer overflows can happen.

The latter makes memory allocations for the MEM_ROOT about 10x more
expensive, so let's always start with an empty memroot (= zero overhead
if no user variables are used) and on THD cleanup let's retain one
memroot block (= zero overhead if the next connection takes THD from the
cache and uses user variables up to one block size).

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-23486 RBR can bypass secure_timestamp=YES

Add tests for system versioning.
INSERT is fixed, but UPDATE is still broken.
Abhishek Bansal
MDEV-38033: JSON_SCHEMA_VALID returns wrong result for array of objects

(11.4 backport from 12.3)

Fix a bug in Json_schema_items::validate where it was missing a
json_skip_level() call. Without this skip, the validator would
incorrectly recurse into non-scalar array elements (like objects)
and try to validate their internal keys/values against the array's
item schema.
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only root

Store all READ_ONLY sysvar values in the read_only_root

Even though READ_ONLY sysvars are protected, for string variables
it usually means that the pointer cannot be changed. The value it
points to - the string itself - still can be. Let's store all
values of string READ_ONLY sysvars in the read_only_root.

sysvars that point directly into argv are copied to read_only_root.
sysvars that have their values calculated and allocated now
must be explicitly marked with PREALLOCATED to let it know they
have to be free()-d.

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-40337 store user vars in a dedicated memroot

user_var_entry objects and their names have a connection lifetime,
they exist until the connection ends (or is reset), and then they're
all deleted at once. This is exactly the use case for MEM_ROOT,
let's store them there.

Additionally, let's set MY_ROOT_USE_MPROTECT flag to keep this memroot
off the general heap where user_var_entry values are stored
and where heap buffer overflows can happen.

The latter makes memory allocations for the MEM_ROOT about 10x more
expensive, so let's always start with an empty memroot (= zero overhead
if no user variables are used) and on THD cleanup let's retain one
memroot block (= zero overhead if the next connection takes THD from the
cache and uses user variables up to one block size).

Assisted-By: Claude:claude-4.8-opus
Yuchen Pei
MDEV-31182 [wip] Drop column stats on incompatible changes

ALTER TABLE ... MODIFY ... should drop column stats when the changes
are incompatible for the purpose of preserving the stats. Previously
this was only done when the alter algorithm is not COPY, and the check
is can_convert_no_copy, which checks compatibility for inplace change
rather than column stats. So we fix that by checking
type_handler_for_comparison equality which seems to suggest stats
compatibility (?).

TODO:
- check whether type_handler_for_comparison equality is sufficient
- check whether affected index stats need to be dropped too
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only segment

Protect all READ_ONLY sysvars from run-time changes.

Put them into a separate ELF section with a special attribute
and mprotect() this section after the server is fully initialized.

Verify that they're all protected in the sys_var constructor.

One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables

On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-37951 SHOW TABLES allows users with only GRANT OPTION privilege to read all table names in the database "mysql"

GRANT_ACL bleeds into thd->col_access via check_access()'s
"*save_priv |= db_access" path (db_access inherits master_access).
Two guards that relied on the absence of any privilege bit then
failed to treat GRANT_ACL as exceptional:

1. check_show_access() (sql_parse.cc): the fallback
  "!col_access && check_grant_db()" was bypassed when col_access
  held only GRANT_ACL, letting a global-GRANT-OPTION user pass the
  outer gate for SHOW TABLES / SHOW TABLE STATUS / SHOW TRIGGERS.
  Fix: use !(col_access & ~GRANT_ACL) so any real privilege still
  bypasses check_grant_db while GRANT_ACL alone does not.

2. get_all_tables() (sql_show.cc): the per-table shortcut
  "!(col_access & TABLE_ACLS)" was taken for col_access == GRANT_ACL
  because GRANT_ACL is part of TABLE_ACLS, skipping per-table checks
  and exposing all table names.
  Fix: use TABLE_ACLS & ~GRANT_ACL so that only genuine table-access
  bits suppress the per-table privilege check.

Assisted-By: Claude:claude-4.6-sonnet
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only segment

Protect all READ_ONLY sysvars from run-time changes.

Put them into a separate ELF section with a special attribute
and mprotect() this section after the server is fully initialized.

Verify that they're all protected in the sys_var constructor.

One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables

On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
MDEV-40341 store read-only sysvars in a read-only root

Store all READ_ONLY sysvar values in the read_only_root

Even though READ_ONLY sysvars are protected, for string variables
it usually means that the pointer cannot be changed. The value it
points to - the string itself - still can be. Let's store all
values of string READ_ONLY sysvars in the read_only_root.

sysvars that point directly into argv are copied to read_only_root.
sysvars that have their values calculated and allocated now
must be explicitly marked with PREALLOCATED to let it know they
have to be free()-d.

Assisted-By: Claude:claude-4.8-opus
Sergei Golubchik
fixup! MDEV-40341 store read-only sysvars in a read-only segment
Abhishek Bansal
MDEV-38033: JSON_SCHEMA_VALID returns wrong result for array of objects

(11.4 backport from 12.3)

Fix a bug in Json_schema_items::validate where it was missing a
json_skip_level() call. Without this skip, the validator would
incorrectly recurse into non-scalar array elements (like objects)
and try to validate their internal keys/values against the array's
item schema.
Aleksey Midenkov
MDEV-17613 Read sole now-partition for non-historical versioned query

When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in the
current partition. Restrict read_partitions to that partition in
vers_setup_conds() instead of relying on a row_end condition.

read_partitions is reset to all partitions every statement in
open_table(), so this restriction must be re-applied on each execution.
Guard it with !vers_conditions.was_set() rather than !is_set():
set_all() sets type (flips is_set()) but not orig_type (was_set()), so
is_set() would stay true across prepared-statement / stored-procedure
re-executions and skip the restriction, leaving history partitions
readable - rows matched then grow on each execute/call.

EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.
ParadoxV5
MDEV-40298 Use-After-Free when SQL Thread stops in the middle of SHOW SLAVE STATUS

MDEV-36287 acknowledged that SHOW SLAVE STATUS needed to acquire a mutex
lock before accessing the SQL Thread’s THD, as otherwise it may access
invalid memory if a concurrent STOP SLAVE deletes the THD.
But it missed that the SQL Thread’s mutex is `mi->rli.run_lock`,
not `mi->run_lock`, so the bug was still not fixed.

This commit fills in the oversight by acquiring the correct
corresponding lock for each of `Slave_IO_State` & `Slave_SQL_State`.

Reviewed-by: Kristian Nielsen <[email protected]>
Daniel Black
MDEV-39742 11.4 JSON functions not interuptable

The JSON functions added in 11.4 where not interuptable
with a KILL QUERY or exceeding the max_statement_time.

* JSON_ARRAY_INTERSECT
* JSON_OBJECT_FILTER_KEYS
* JSON_OBJECT_TO_ARRAY
* JSON_SCHEMA_VALID

This behaviour is corrected by setting the killed_ptr
of the json_engine_t structure.

JSON_KEY_VALUE was added in 11.4, however as it doesn't call
json_next_value(), its processing doesn't check the killed pointer
of the json engine. So its quick anyway.

As a result of added error handling, some warnings are generated.
Daniel Black
merge - json_normalize changes
Daniel Black
MDEV-39742 11.4 JSON functions not interuptable

The JSON functions added in 11.4 where not interuptable
with a KILL QUERY or exceeding the max_statement_time.

* JSON_ARRAY_INTERSECT
* JSON_OBJECT_FILTER_KEYS
* JSON_OBJECT_TO_ARRAY
* JSON_SCHEMA_VALID

This behaviour is corrected by setting the killed_ptr
of the json_engine_t structure.

JSON_KEY_VALUE was added in 11.4, however as it doesn't call
json_next_value(), its processing doesn't check the killed pointer
of the json engine. So its quick anyway.

As a result of added error handling, some warnings are generated.
Georg Richter
Remove MySQL specific test
Daniel Black
merge - json_normalize changes