Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Aleksey Midenkov
MDEV-17613 Test cleanup: sorted result for versioning.partition

Avoid result fluctuations with different storage engines.
Yuchen Pei
MDEV-41366 Check prefix key match in iterated partition index scan

The idea of Case 2 in can_skip_merging_scans is that when partition
column key prefix is fixed, and the infix that is also the "partition
by range" column, we can scan each partition in order. This relies on
accurately returning EOF when a partition has no (more) matching rows.
It is possible to have no more matching rows at the first index read
of a partition in a non-exct access, in which case we need to check
the prefix match to rule out false positives and return EOF correctly.

This change by itself would result in incomplete results, if Case 2
incorrectly determines the scan can iterate over partitions in order
in AFTER_KEY and BEFORE_KEY reads. These reads when applied to a
partition column key prefix, necessarily means seeking a different
prefix value. This could result in false negatives. To that end, we
strengthen the checks in Case 2 so that AFTER_KEY and BEFORE_KEY reads
require both the prefix and the partitioned column itself in the
keypart_map to ensure correctness.

For example, this check disqualifies the index_read_map call in an
existing loose index scan testcase, causing the index scan to use both
unordered (in read_range_first) and ordered.
Sergei Golubchik
Merge branch '11.4' into 11.8
Sergei Golubchik
JSON data type pre-feature

pretend better that JSON is a proper data type:

* SHOW CREATE shows JSON for LONGTEXT CHARSET utf8mb4_bin
  CHECK(IS_JSON_VALID()) columns.
* CAST(... AS JSON) is accepted (casts to TEXT CHARSET utf8mb4_bin, with
  no real JSON validation)

These two placeholders will be fully correct when MDEV-38740 is pushed.
Sergei Golubchik
cleanup: extract duplicated no_part_keypart() check
Sergei Golubchik
HA_ERR_NULL_IN_SPATIAL should be ignorable error

index or column that accepts only certain values is like an implicit
CHECK constraint and should behave similarly. It even can be shielded
with, say, `CHECK (length(gis_col)>0)` which will make the error
ignorable. So it should be ignorable without `CHECK` too.
Khaled Riyad
MDEV-31567 SIGSEGV in reinit_stmt_before_use | Prepared_statement::execute

If the first execution of a prepared statement was killed (for
example by max_statement_time) after the top-level JOIN had done the
once-per-statement semi-join rewrite of the nested subqueries, but
before the nested selects saved their rewritten WHERE into prep_where,
the statement was left half-transformed. A later execution then used
a freed Item from prep_where or an unfixed Item_field and crashed.

Set LEX::needs_reprepare when JOIN::optimize() fails during the
select's first optimization, so the statement is re-prepared before
it runs again.
Aleksey Midenkov
MDEV-17613 Refactor prune_partitions() to build into a dedicated bitmap

prune_partitions() built the used-partition set directly in
read_partitions: it cleared it, let find_used_partitions*() fill it, and
in the index-merge case used it as scratch too - both buffer and result.

The patch adds PART_PRUNE_PARAM::parts_bitmap and builds the return
into it instead. read_partitions is produced from it only at the end:
copied when the condition was analyzed (IMPOSSIBLE included,
parts_bitmap left empty), or set to all locked partitions via
mark_all_partitions_as_used() when nothing could be pruned.

Pure refactoring: read_partitions ends up identical on every path. It
decouples the pruning scratch from read_partitions, so a later commit can
switch the final bitmap_copy() to bitmap_intersect() - letting pruning
narrow a pre-set read_partitions (e.g. a versioned table selecting only
its current partition) instead of overwriting it.
Aleksey Midenkov
MDEV-17613 Read sole now-partition for non-historical versioned query

When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in the
current partition. Restrict read_partitions to that partition in
vers_setup_conds() instead of relying on a row_end condition.

read_partitions is reset to all partitions every statement in
open_table(), so this restriction must be re-applied on each execution.
Guard it with !vers_conditions.was_set() rather than !is_set():
set_all() sets type (flips is_set()) but not orig_type (was_set()), so
is_set() would stay true across prepared-statement / stored-procedure
re-executions and skip the restriction, leaving history partitions
readable - rows matched then grow on each execute/call.

A query whose time comes from @@system_versioning_asof must still read
history, so the restriction is also skipped when from_sysvar is set.
init_from_sysvar() sets this new flag rather than orig_type, so
was_set(), print() and the write lock check still see only an explicit
FOR SYSTEM_TIME. Like orig_type, the flag persists to prepared-statement
re-execution.

EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.

With sql_safe_updates, DELETE without WHERE on such a table is now
rejected even with LIMIT, as for a non-versioned table.
Aleksey Midenkov
MDEV-17613 Refactor prune_partitions() to build into a dedicated bitmap

prune_partitions() built the used-partition set directly in
read_partitions: it cleared it, let find_used_partitions*() fill it, and
in the index-merge case used it as scratch too - both buffer and result.

The patch adds PART_PRUNE_PARAM::parts_bitmap and builds the return
into it instead. read_partitions is produced from it only at the end:
copied when the condition was analyzed (IMPOSSIBLE included,
parts_bitmap left empty), or set to all locked partitions via
mark_all_partitions_as_used() when nothing could be pruned.

Pure refactoring: read_partitions ends up identical on every path. It
decouples the pruning scratch from read_partitions, so a later commit can
switch the final bitmap_copy() to bitmap_intersect() - letting pruning
narrow a pre-set read_partitions (e.g. a versioned table selecting only
its current partition) instead of overwriting it.
Aleksey Midenkov
MDEV-17613 Test cleanup: sorted result for versioning.partition

Avoid result fluctuations with different storage engines.
Aleksey Midenkov
MDEV-17613 Refactor prune_partitions() to narrow read_partitions instead of full rewrite

read_partitions is reset from lock_partitions once per statement in
open_table() (set_partition_bitmaps()), and prune_partitions() only
intersects it with the pruned set instead of rebuilding it.
mark_all_partitions_as_used() is dropped.

Pure refactoring, results unchanged. It lets a restriction set on
read_partitions before pruning survive it - needed when read_partitions
is pre-restricted, e.g. a versioned table whose history is partitioned
reads only the current partition.
Aleksey Midenkov
MDEV-17613 bitmap_range() for iteration over partition bitmap

Returns bit range for partition element.
Sergei Golubchik
cleanup: const, spaces, comments
Sergei Golubchik
cleanup: remove redundant function
Aleksey Midenkov
MDEV-17613 Refactor prune_partitions() to build into a dedicated bitmap

prune_partitions() built the used-partition set directly in
read_partitions: it cleared it, let find_used_partitions*() fill it, and
in the index-merge case used it as scratch too - both buffer and result.

The patch adds PART_PRUNE_PARAM::parts_bitmap and builds the return
into it instead. read_partitions is produced from it only at the end:
copied when the condition was analyzed (IMPOSSIBLE included,
parts_bitmap left empty), or set to all locked partitions via
mark_all_partitions_as_used() when nothing could be pruned.

Pure refactoring: read_partitions ends up identical on every path. It
decouples the pruning scratch from read_partitions, so a later commit can
switch the final bitmap_copy() to bitmap_intersect() - letting pruning
narrow a pre-set read_partitions (e.g. a versioned table selecting only
its current partition) instead of overwriting it.
ParadoxV5
MDEV-40271: Disallow `\0` and `\n` in replication connection names

Null bytes conflict with strings’ null-termination,
\while newline bytes are delimiters in the `multi-master.info` file.
While filename-unsafe chars are canonicalized when mapping names
to filenames, these two still cause trouble in other places.
Therefore, this commit extends `check_master_connection_name()`
to also reject names containing these chars.

This check function is now also used for system variable basenames
(e.g., in `@@replicate_do_db`), which (probably harmlessly)
did not have the existing `MAX_CONNECTION_NAME` check.
Aleksey Midenkov
MDEV-17613 Refactor prune_partitions() to narrow read_partitions instead of full rewrite

read_partitions is reset to all partitions (or the PARTITION() set) once
per statement in open_table() (set_partition_bitmaps()), and
prune_partitions() only intersects it with lock_partitions and the pruned
set instead of rebuilding it.
mark_all_partitions_as_used() is dropped.

Pure refactoring, results unchanged. It lets a restriction set on
read_partitions before pruning survive it - needed when read_partitions
is pre-restricted, e.g. a versioned table whose history is partitioned
reads only the current partition.
Aleksey Midenkov
MDEV-17613 bitmap_range() for iteration over partition bitmap

Returns bit range for partition element.
Sergei Golubchik
cleanup: is_local_field() -> get_local_field()

removes the need for a separate cast after the check,
and guarantees that the correct item is used as Item_field
(and not something else was mistakenly cast due to a typo)
Sergei Petrunia
MDEV-36080: Assertion on 2nd PS execution with error and Array Binding

The fix for MDEV-35318 has introduced LEX::needs_reprepare and logic in
Prepared_statement::execute_loop() and sp_lex_keeper::
validate_lex_and_exec_core() to re-prepare the statement if it has hit
an error when doing once-per-statement-life optimizations.

But there is also third code path: PS with Array Binding is handled in
Prepared_statement::execute_bulk_loop(). Add handling there as well.

(cherry picked from commit 33e0796e7a154e02a5e53c55cefc5d6feb4f5e6d)
Sergei Golubchik
cleanup: remove obsolete code
Aleksey Midenkov
MDEV-17613 bitmap_range() for iteration over partition bitmap

Returns bit range for partition element.
Vladislav Vaintroub
MDEV-28746 follow-up: missed relative paths overflowing MAX_PATH

my_win_sopen() only checked the length of the raw path it was given, but
the server opens table files via paths relative to its data directory,
so a short relative path can still overflow MAX_PATH once
resolved against a long enough datadir. That case was misreported as
ER_BAD_DB_ERROR instead of ER_CANT_CREATE_TABLE/ENAMETOOLONG.

Resolve the path via GetFullPathName() before deciding whether the path
was too long, factored into resolved_path_too_long(). A failed resolve
is only treated as too-long when GetLastError() is
ERROR_FILENAME_EXCED_RANGE, so unrelated resolution failures keep
mapping to the original CreateFile error.

Also fix the global errtable mapping of ERROR_FILENAME_EXCED_RANGE: it
was mapped to ENOENT, so any Win32 call that got that error code
directly (not via ERROR_PATH_NOT_FOUND) was misreported the same way.

Assisted-by: Claude:claude-sonnet-5
Sergei Golubchik
cleanup: remove dead code
Sergei Golubchik
generalize HA_ERR_NULL_IN_SPATIAL

* make it mean "tthe index rejects this field value"
* rename to HA_ERR_BAD_FIELD_VALUE (not user-visible)
* change handler::print_error() to provide a more helpful error message

-Warning 1416 Cannot get geometry object from data you send to the GEOMETRY field
+Warning 1366 Incorrect geometry value: '' for column `test`.`t1`.`b` at row 2

  at least for cases when get_dup_key() returns a valid affected key,
  otherwise fallback to the old error message.
ParadoxV5
MDEV-40271: Disallow `\0` and `\n` in replication connection names

Null bytes conflict with strings null-termination,
\while newline bytes are delimiters in the `multi-master.info` file.
While filename-unsafe chars are canonicalized when mapping names
to filenames; these two still cause trouble in other places.
Therefore, this commit extends `check_master_connection_name()`
to also reject names containing these chars.

This check function is now also used for system variable basenames
(e.g., in `@@replicate_do_db`), which (probably harmlessly)
did not have the existing `MAX_CONNECTION_NAME` check.
Sergei Golubchik
support AND/OR ranges over spatial indexes

* remove the code that unconditionally disabled indexes when GEOM_FLAG
  was present on ranges
* instead, treat two spatial ranges as two _separate index scans_,
  unlike normal indexes, two spatial ranges cannot be combined.
  Optimizer later picks one with the better selectivity.
* this means there can be up to MAX_KEY ranges (SEL_TREE) even if the
  table has only one or two indexes - limit all arrays by MAX_KEY, not
  by param->keys.
* the KEY_PART is cloned as needed to create separate spatial ranges
Aleksey Midenkov
MDEV-17613 Test cleanup: sorted result for versioning.partition

Avoid result fluctuations with different storage engines.
Sergei Golubchik
cleanup: change mhnsw_read_first() API to take the value not Item*
Sergei Golubchik
Tree<> - a typesafe wrapper for TREE
Sergei Golubchik
cleanup: KEY::type() and KEY::is_hlindex() methods
Sergei Golubchik
MDEV-25848 Support for Multi-Valued Indexes

* New type of high-level indexes - JSON ARRAY
* sql/index/json.{h,cc}
  * table structure (value varbinary(255), tref varbinary(N), PK(value,tref))
  * insert/update/delete/search works, search uses keyread
  * on insert json array is parsed, values are inserted
    * prefixed by the type tag, only simple types are supported
  * records_in_range works, but only for equalities (MEMBER OF)
* MySQL-compatible syntax `INDEX ((CAST(col AS type ARRAY)))`, but
  * col is a bare name, not an expression, expressions are part of MDEV-35853
  * type is ignored, not stored, also part of MDEV-35853
  * thus `'[1, 0.2, true, "foo", null]'` works fine, type not enforced
  * thus SHOW CREATE TABLE prints `... AS JSON ARRAY`
* `const MEMBER OF (col)` uses index, via
  * range optimizer, QUICK_RANGE_SELECT_ARRAY
  * Item_func_member_of::get_mm_tree()/etc
Aleksey Midenkov
MDEV-17613 Read sole now-partition for non-historical versioned query

When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in the
current partition. Restrict read_partitions to that partition in
vers_setup_conds() instead of relying on a row_end condition.

read_partitions is reset to all partitions every statement in
open_table(), so this restriction must be re-applied on each execution.
Guard it with !vers_conditions.was_set() rather than !is_set():
set_all() sets type (flips is_set()) but not orig_type (was_set()), so
is_set() would stay true across prepared-statement / stored-procedure
re-executions and skip the restriction, leaving history partitions
readable - rows matched then grow on each execute/call.

A query whose time comes from @@system_versioning_asof must still read
history, so the restriction is also skipped when from_sysvar is set.
init_from_sysvar() sets this new flag rather than orig_type, so
was_set(), print() and the write lock check still see only an explicit
FOR SYSTEM_TIME. Like orig_type, the flag persists to prepared-statement
re-execution.

EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.

With sql_safe_updates, DELETE without WHERE on such a table is now
rejected even with LIMIT, as for a non-versioned table.
Aleksey Midenkov
MDEV-17613 Read sole now-partition for non-historical versioned query

When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in the
current partition. Restrict read_partitions to that partition in
vers_setup_conds() instead of relying on a row_end condition.

read_partitions is reset to all partitions every statement in
open_table(), so this restriction must be re-applied on each execution.
Guard it with !vers_conditions.was_set() rather than !is_set():
set_all() sets type (flips is_set()) but not orig_type (was_set()), so
is_set() would stay true across prepared-statement / stored-procedure
re-executions and skip the restriction, leaving history partitions
readable - rows matched then grow on each execute/call.

A query whose time comes from @@system_versioning_asof must still read
history, so the restriction is also skipped when from_sysvar is set.
init_from_sysvar() sets this new flag rather than orig_type, so
was_set(), print() and the write lock check still see only an explicit
FOR SYSTEM_TIME. Like orig_type, the flag persists to prepared-statement
re-execution.

EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.

With sql_safe_updates, DELETE without WHERE on such a table is now
rejected even with LIMIT, as for a non-versioned table.
Sergei Golubchik
skip hlindex update if its columns aren't in the write_set
Sergei Petrunia
MDEV-35318 Assertion `tl->jtbm_subselect' failed in JOIN::calc_allowed_top_level_tables

Alternative, more general fix, Variant 2.

The problem was as follows: Suppose we are running a PS/SP statement and
we get an error while doing optimization that is done once per statement
life. This may leave the statement data structures in an undefined state,
where it is not safe to execute it again.

The fix: introduce LEX::needs_reprepare and set it in such cases.
Make PS and SP runtime check it and re-prepare the statement before
executing it again.

We do not use Reprepare_observer, because it turns out it is tightly tied
to watching versions of statement's objects. For example, it must not be
used when running the statement for the first time, exactly when the
once-per-statement-lifetime optimizations are done.

(cherry picked from commit 0e21ff8ca47ab637564b4c120322215caa70835d)
Sergei Petrunia
MDEV-36057: Assertion failure on 2nd execution of parameterized PS

Followup to fix for MDEV-35318: In Prepared_statement::execute_loop(),
if we enter this function with lex->needs_reprepare=true, we need to
re-prepare the statement.
We also need to call set_parameters() to get PS parameter values. We
failed to do this and that's the bug.

Note that we need to also handle the other reprepare scenario: when
we get into Prepared_statement::execute_loop(), call set_parameters()
and then hit a "DDL was changed" error with reprepare_observer,
we should NOT call set_parameters() again.
The parameters have already been transferred to the reprepared statement
in Prepared_statement::reprepare()|swap_parameter_array().

(cherry picked from commit 80067a69feaeb5df30abb1bfaf7d4e713ccbf027)
Aleksey Midenkov
MDEV-17613 Refactor prune_partitions() to narrow read_partitions instead of full rewrite

read_partitions is reset to all partitions (or the PARTITION() set) once
per statement in open_table() (set_partition_bitmaps()), and
prune_partitions() only intersects it with lock_partitions and the pruned
set instead of rebuilding it.
mark_all_partitions_as_used() is dropped.

Pure refactoring, results unchanged. It lets a restriction set on
read_partitions before pruning survive it - needed when read_partitions
is pre-restricted, e.g. a versioned table whose history is partitioned
reads only the current partition.
Sergei Golubchik
cleanup: move mhnsw behind the hlindex interface

* new hlindex home: sql/index/
* API classes:
  * hlindexton - singleton, global methods. it creates
  * hlindex_share - a physical index (like TABLE_SHARE), it creates
  * hlindex - one index cursor (like handler or TABLE)
* TABLE stores hlindex, TABLE_SHARE stores hlindex_share, no more void*
* move vector_hnsw.h to its new home sql/index, added sql/index/hlindex.h
* MHNSW_Share is stored in mhnsw_share, Search_context in mhnsw_index
* KEY::options() method