Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Geng Tian
MDEV-39995 JSON_CONTAINS and JSON_EQUALS do not compare strings based on semantic

JSON_CONTAINS, JSON_EQUALS, and JSON_OVERLAPS used raw byte-level
comparison (memcmp) for JSON string values, which meant semantically
equivalent strings like "A" and "\u0041" were incorrectly treated
as different.

Fix: add json_string_compare() that decodes Unicode escape sequences
before comparing, and fix json_normalize to produce a canonical form
for strings with escapes so JSON_EQUALS works correctly.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Georg Richter
Windows fix
ParadoxV5
MDEV-40298 Use-After-Free when SQL Thread stops in the middle of SHOW SLAVE STATUS

MDEV-36287 acknowledged that SHOW SLAVE STATUS needed to acquire a mutex
lock before accessing the SQL Thread’s THD, as otherwise it may access
invalid memory if a concurrent STOP SLAVE deletes the THD.
But it missed that the SQL Thread’s mutex is `mi->rli.run_lock`,
not `mi->run_lock`, so the bug was still not fixed.

This commit fills the oversight in by acquiring the correct
corresponding lock for each of `Slave_IO_State` & `Slave_SQL_State`.
Georg Richter
Fix build
Georg Richter
Fix MySQL error: replace SLAVE by REPLICA
Georg Richter
Change post_header_len to signed (ssize_t)
Sergei Golubchik
MDEV-40362 #mysql50# mixes up with table names in the table cache and on disk

Only allow #mysql50# prefix on table names that would've been
otherwise encoded. It is an error to put this prefix on a table
that maps to the file name without a conversion.

Don't allow creation of new tables with the #mysql50# prefix.

Except when in SQLCOM_ALTER_DB_UPGRADE - main.upgrade test runs
ALTER DATABASE `#mysql50#mysqltest-1` UPGRADE DATA DIRECTORY NAME;
This command moves tables from the old db name to the new db name,
and some of these tables also have old #mysql50# names.
PranavKTiwari
Added check.
Georg Richter
Fix build errors
Daniel Black
MDEV-40165 10.11 JSON functions don't respect KILL QUERY

Or max_statement_time limit.

Implement interuptability of:
* JSON_OVERLAPS
* JSON_LENGTH
* JSON_DEPTH

As json_report_error now processes the killed there's
no need of thd->check_killed(). Removed this from a few
locations.

Verified that the path after every json_scan_next, the
location of checking the killed_ptr, reports errors
correctly.
Georg Richter
Fix windows build error (type conversion)
Aleksey Midenkov
MDEV-17613 bitmap_range() refactoring
Daniel Black
MDEV-40175: JSON_VALID don't respect KILL QUERY

Or max_statement_time limit.

Change the json_valid to take a json_engine_t argument.
Adjust the Item_json_valid to have a json_engine_t and
to report an error.

Note this means that invalid json syntax now has a
note on any json error. Raising this to a warning can
cause check constraints of JSON_VALID() to result
in ER_JSON_SYNTAX rather than ER_CONSTRAINT_FAILED
(valided in main.type_json test).

Consequential changes in debug build to json acl
testing validation and unit test having a dummy json_engine_t.

json_normalize also uses the same engine that was
passed as its argument to test json_valid. Because
the character set may have changed, if there's an error
adjust the je->s.c_str pointer. Its only the offset that
is currently used in the error messages anyway so there
may be offset errors depending on the original character
set.
Georg Richter
Build fix: Fix signed/unsigned warning
Georg Richter
Added additional check for post_header_lengths

If a format description event was processed, we calculate
the post_header_len for the specific event at the beginning
of mariadb_rpl_fetch and check for under- or overflow
Alexander Barkov
MDEV-39022 Add `LOCAL spvar` syntax for prepared statements and SYS_REFCURSORs

This patch adds the following syntax:

OPEN c0 FOR LOCAL spvar_with_ps_name;
PREPARE LOCAL spvar_with_ps_name FROM 'dynamic sql';
EXECUTE LOCAL spvar_with_ps_name;
DEALLOCATE PREPARE LOCAL spvar_with_ps_name;

OPEN c0 FOR PREPARE stmt;
Aleksey Midenkov
MDEV-17613 Refactor partition pruning to build into a dedicated bitmap

prune_partitions() built the used-partition set directly in
read_partitions: it cleared it, let find_used_partitions*() fill it, and
in the index-merge case used it as scratch too - both buffer and result.

The patch adds PART_PRUNE_PARAM::parts_bitmap and builds the return
into it instead. read_partitions is produced from it only at the end:
copied when the condition was analyzed (IMPOSSIBLE included,
parts_bitmap left empty), or set to all locked partitions via
mark_all_partitions_as_used() when nothing could be pruned.

Pure refactoring: read_partitions ends up identical on every path. It
decouples the pruning scratch from read_partitions, so a later commit can
switch the final bitmap_copy() to bitmap_intersect() - letting pruning
narrow a pre-set read_partitions (e.g. a versioned table selecting only
its current partition) instead of overwriting it.
Aleksey Midenkov
MDEV-17613 prune_partitions() narrows read_partitions instead of rewrite

read_partitions is reset from lock_partitions once per statement in
open_table() (set_partition_bitmaps()), and prune_partitions() only
intersects it with the pruned set instead of rebuilding it.
mark_all_partitions_as_used() is dropped.

Pure refactoring, results unchanged. It lets a restriction set on
read_partitions before pruning survive it - needed when read_partitions
is pre-restricted, e.g. a versioned table whose history is partitioned
reads only the current partition.
Yuchen Pei
MDEV-31182 Check for blob too in create_key_parts_for_pseudo_indexes

BLOB and VARCHAR has keys prefixed with length of the payload. This
length data itself has length HA_KEY_BLOB_LENGTH which is 2, as
demonstrated for example in Field_blob::set_key_image and
Field_varstring::set_key_image.

In this patch, when preparing key info for a range analysis, we
account for this extra HA_KEY_BLOB_LENGTH in the total key length for
BLOB. Previously it was only done for VARCHAR. This way, when doing
the actual range analysis, LIKE (Item_func_like::get_mm_leaf) can
correctly identify that it is dealing with a BLOB, thereby allowing
the space of 2 for the length data. This avoids later
Field_blob::set_key_image reading the payload as length data and
causing reading the wrong memory segment for the payload.
Georg Richter
MySQL 9.7 fix: Don't use MASTER STATUS
Yuchen Pei
MDEV-31182 Make Field_geom::set_key_image do nothing

In fact, it should not be called at all, as GIS indexes do not store
the whole data.

Remove the only known callsite covered by a testcase (added in this
commit), in print_key_part_value

The new testcase results in new coverage that exposes an MSAN
uninitialised failure, where range->end_key.keypart_map is not
assigned for GIS indexes in sel_arg_range_seq_next, so we accordingly
do not use that keypart_map in the subsequent print_range call
Daniel Black
MDEV-31554 Cursor protocol cuts off json boolean

In Item_func_json_array_append::fix_length_and_dec,
Item_bool::max_char_length() was being used to
calculate the length that is required to fit a boolean.

Previously it was returning 1, which doesn't fit
"false".

Change Item_bool to return strlen("false"), 5.
Alexey (Holyfoot) Botchkov
MDEV-40013 Server crashes in XMLSchema_user_type::validate_prepare/XMLSchema_attribute::validate_prepare.

complexType doesn't have to have any compositor at all. So let's set
'empty_compositor' in this case.
Yuchen Pei
MDEV-31182 Check for blob too in create_key_parts_for_pseudo_indexes

BLOB and VARCHAR has keys prefixed with length of the payload. This
length data itself has length HA_KEY_BLOB_LENGTH which is 2, as
demonstrated for example in Field_blob::set_key_image and
Field_varstring::set_key_image.

In this patch, when preparing key info for a range analysis, we
account for this extra HA_KEY_BLOB_LENGTH in the total key length for
BLOB. Previously it was only done for VARCHAR. This way, when doing
the actual range analysis, LIKE (Item_func_like::get_mm_leaf) can
correctly identify that it is dealing with a BLOB, thereby allowing
the space of 2 for the length data. This avoids later
Field_blob::set_key_image reading the payload as length data and
causing reading the wrong memory segment for the payload.
Georg Richter
Check if we are running against MySQL or MariaDB
Aleksey Midenkov
MDEV-17613 prune_partitions() narrows read_partitions instead of rewrite

read_partitions is reset from lock_partitions once per statement in
open_table() (set_partition_bitmaps()), and prune_partitions() only
intersects it with the pruned set instead of rebuilding it.
mark_all_partitions_as_used() is dropped.

Pure refactoring, results unchanged. It lets a restriction set on
read_partitions before pruning survive it - needed when read_partitions
is pre-restricted, e.g. a versioned table whose history is partitioned
reads only the current partition.
Alexey (Holyfoot) Botchkov
MDEV-40013 Server crashes in XMLSchema_user_type::validate_prepare/XMLSchema_attribute::validate_prepare.

complexType doesn't have to have any compositor at all. So let's set
'empty_compositor' in this case.
Aleksey Midenkov
MDEV-17613 Read sole now-partition for non-historical versioned query

When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in the
current partition. Restrict read_partitions to that partition in
vers_setup_conds() instead of relying on a row_end condition.

read_partitions is reset to all partitions every statement in
open_table(), so this restriction must be re-applied on each execution.
Guard it with !vers_conditions.was_set() rather than !is_set():
set_all() sets type (flips is_set()) but not orig_type (was_set()), so
is_set() would stay true across prepared-statement / stored-procedure
re-executions and skip the restriction, leaving history partitions
readable - rows matched then grow on each execute/call.

EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.
Sergei Golubchik
MDEV-40362 #mysql50# mixes up with table names in the table cache and on disk

Only allow #mysql50# prefix on table names that would've been
otherwise encoded. It is an error to put this prefix on a table
that maps to the file name without a conversion.

Don't allow creation of new tables with the #mysql50# prefix.

Except when in SQLCOM_ALTER_DB_UPGRADE - main.upgrade test runs
ALTER DATABASE `#mysql50#mysqltest-1` UPGRADE DATA DIRECTORY NAME;
This command renames all tables from the old db name to the new db name,
and some of these tables also have old #mysql50# names.
Thirunarayanan Balathandayuthapani
MDEV-40332 InnoDB: Defragmentation of BASE_IDX in SYS_VIRTUAL failed: Data structure corruption

Problem:
========
- While shrinking the system tablespace, InnoDB defragments the
system tables to move used pages out of the extents
near the end of the file so the tail can be truncated.
defragment_level() relocates every page of a to-be-moved extent
and, to do so, rewrites the node pointer in the page's parent.

A B-tree root page has no parent node pointer, so it is
never recorded in m_parent_pages.
When the root of a system-table index happens to reside in an extent
that was selected for relocation, the parent lookup fails and
defragmentation is aborted with DB_CORRUPTION even though nothing is
corrupt.

Solution:
=========
  Because a root page cannot move, the system tablespace cannot
shrink below the highest root page, and relocating any
page at or below it cannot reduce the file size.
Exclude that region from relocation up front.

SpaceDefragmenter::max_root_extent(): New function returning the
highest extent that holds a root page of a system-table index

SpaceDefragmenter::find_new_extents(): Use max_root_extent()
together with the minimum tablespace size as the lower bound
(floor) of the relocation scan, so no extent at or below the
highest root is ever added to the relocation map.
This avoids the false corruption and also avoids reserving a
destination extent for a move that would never happen.
The free tail above the highest root is still reclaimed by truncation.
Daniel Black
Merge 10.11 into 11.4
Yuchen Pei
MDEV-31182 Make Field_geom::set_key_image do nothing

In fact, it should not be called at all, as GIS indexes do not store
the whole data.

Remove the only known callsite covered by a testcase (added in this
commit), in print_key_part_value

The new testcase results in new coverage that exposes an MSAN
uninitialised failure, where range->end_key.keypart_map is not
assigned for GIS indexes in sel_arg_range_seq_next, so we accordingly
do not use that keypart_map in the subsequent print_range call
Daniel Black
MDEV-40165: JSON_EQUAL/JSON_NORMALIZE dont respect KILL QUERY

Or max_statement_time limit.

JSON_EQUAL/JSON_NORMALIZE also didn't report warnings
on invalid JSON inputs.

Change the json_normalize function to take a json_engine_t
as a argument and implement the sql/item_jsonfunc.cc calls
to have a json_engine_t.

To the JSON_EQUALS and JSON_NORMALIZE, add error handling
to produce an error when there was one. compare_nested_object,
part of JSON_OVERLAPS, uses a new json_engine and
copies the error/position to the returning function.

json_normalize expects the incoming je structure to have
a valid, or nullptr, killed_ptr, as json_start will reset this.
Yuchen Pei
MDEV-31182 Fix an MSAN issue introduced in the parent commit

(this commit is split out for a targeted review that does not
duplicate the review of its parent commit)

The new testcase results in new coverage that exposes an MSAN
uninitialised failure, where range->end_key.keypart_map is not
assigned for GIS indexes in sel_arg_range_seq_next, so we accordingly
do not use that keypart_map in the subsequent print_range call
Alexander Barkov
MDEV-39022 Add `LOCAL spvar` syntax for prepared statements and SYS_REFCURSORs

This patch adds the following syntax:

OPEN c0 FOR LOCAL spvar_with_ps_name;
PREPARE LOCAL spvar_with_ps_name FROM 'dynamic sql';
EXECUTE LOCAL spvar_with_ps_name;
DEALLOCATE PREPARE LOCAL spvar_with_ps_name;

OPEN c0 FOR PREPARE stmt;
Alexey (Holyfoot) Botchkov
MDEV-40361 Server crashes after select xmlisvalid (validate_prepare/base).

Type definitions have to be checked on circular references.
Alexey (Holyfoot) Botchkov
MDEV-40357 Server crashes in XMLSchema_group_reference::validate_prepare.

The <group> in the XML Schema can legally be empty. Handle that case.
Yuchen Pei
MDEV-31182 Make Field_geom::set_key_image do nothing

In fact, it should not be called at all, as GIS indexes do not store
the whole data.

Remove the only known callsite covered by a testcase (added in this
commit), in print_key_part_value
Rex Johnston
MDEV-36610 Subquery wrongly eliminated by table elimination

When equality propagation (build_equal_items()) merges an equality that
contains a subquery, such as "t1.a = (SELECT ...)", with an outer join's
ON equality, it can inject a reference to that subquery into the ON
expression. If the join columns have compatible types the subquery ends
up as the constant of a multiple equality (which Item::walk() skips); if
they differ (e.g. BIGINT vs INT) the field cannot be merged and the
subquery is substituted in as a plain "tbl.col = (SELECT ...)" argument.

In the latter case, if that outer join is removed by table elimination,
mark_as_eliminated() walks the ON expression and flags the shared
Item_subselect as eliminated. The subquery, however, still lives in
another part of the query and has to be executed, tripping
DBUG_ASSERT(!eliminated) in Item_subselect::exec() (and, in release
builds, disabling the subquery cache and hiding it from EXPLAIN).

The surviving reference can be:
- a WHERE/HAVING/select-list/ORDER/GROUP expression (subquery written
  there and pushed down into the eliminated ON), or
- the ON expression of an outer join that was not eliminated (subquery
  written in a surviving outer ON and pushed down into an eliminated
  inner one).

Fix: after table elimination, walk the expressions that survive into
execution (WHERE, HAVING, select list, ORDER/GROUP BY and the ON
expressions of outer joins that were not eliminated) and clear the
"eliminated" flag on any subquery still reachable from them.

Because a subquery can also be the constant of a multiple equality, and
Item::walk() does not visit an Item_equal's constant, Item_equal gets an
unmark_as_eliminated_processor() override that descends into its constant
explicitly.

Assisted by Claude Opus 4.8