Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Yuchen Pei
MDEV-37330 Allow unordered index scan for tables partitioned by range when a priority queue is not needed

There are two types of partition index scans. Ordered and unordered.
The ordered scan utilises a priority queue of size equal to the number
of partitions. Thus when the number of partitions is large and a
SELECT query with ORDER BY request only a few results (e.g. LIMIT 1),
populating the partitions can be wasteful. Therefore in this patch we
allow the use of unordered scan when a priority queue is not needed.

This includes:

Case 1. The PARTITION BY RANGE expression is a col that is a prefix of
the active index and we are in
index_first/index_last/index_read_map/read_range_first/multi_range_read_next,
OR

Case 2. The PARTITION BY RANGE expression is col1 and the active
index is (prefix_cols, col1, ...), and we are in
index_read_map(prefix_cols=prefix_value), or
read_range_first(start_key= {prefix_value, ...},
end_key={prefix_value, ...}), or
multi_range_read_next(start_key= {prefix_value, ...},
end_key={prefix_value, ...})

Limitation: not supporting reverse index

test passed with:

mtrdefault --suite main --do-test=.*partition
mtrdefault --suite parts
Yuchen Pei
MDEV-37330 Indicate in ANALYZE FORMAT=JSON index scan method used for partitioned tables

There are two types of partition index scans. Ordered and unordered.
Ordered scan is used in common cases, including when the partitioned
table is the first in the join order and when ORDER BY ... DESC.
Whereas unordered scan is used when the partitioned table is not the
first in the join order, or in mrr when mrr_mode does not have
HA_MRR_SORTED. It can be unclear whether ordered or unordered scan has
been used. Therefore in this patch we add a field
"partitions_index_scan_method" to ANALYZE FORMAT=JSON to reveal with
method has been used. Possible values include
"merge_ordered_scans" (for ordered scan)
"iterate_over_partitions" (for unordered scan) and "both" (when both
have been used).
Vladislav Vaintroub
MDEV-37424 main.connect fails sporadically with a diff

Wait for disconnects to really finish before FLUSH GLOBAL STATUS
and testing max_used_connections.

The finished "disconnect" can't be relied on when testing this
variable, it is decremented after the socket is closed on server.
Oleg Smirnov
MDEV-38574 Rename cloning functions of class Item and descendants

Rename cloning methods of class Item and its descendants
in the following way:

  (from)            (to)
do_build_clone  -> deep_copy
  build_clone  -> deep_copy_with_checks

do_get_copy  -> shallow_copy
  get_copy  -> shallow_copy_with_checks

to better reflect their functionality.

Also make Item::deep_copy() and shallow_copy() protected.
Outside users should call deep_copy_with_checks()
and shallow_copy_with_checks().
Vladislav Vaintroub
MDEV-37424 main.connect fails sporadically with a diff

Wait for disconnects to really finish before FLUSH GLOBAL STATUS
and testing max_used_connections.

The finished "disconnect" can't be relied on when testing this
variable, it is decremented after the socket is closed on server.
Vladislav Vaintroub
MDEV-37424 main.connect fails sporadically with a diff

Wait for disconnects to really finish before FLUSH GLOBAL STATUS
and testing max_used_connections.

The finished "disconnect" can't be relied on when testing this
variable, it is decremented after the socket is closed on server.
ParadoxV5
last nits
Oleg Smirnov
MDEV-38574 Rename cloning functions of class Item and descendants

Rename `Item::clone_item()` to `clone_constant()`, and do
the same for any overloads in descendant items.
The function returns non-NULL only for items that represent
constant literals.
Sergei Golubchik
MDEV-38604 Assertion `thd->utime_after_query >= thd->utime_after_lock' failed in query_response_time_audit_notify on 2nd execution of SP with query cache

even when PS is served from a query cache, thd->utime_after_query
must be updated.

also, backport the assert from 11.8
Vladislav Vaintroub
MDEV-37527 Client plugins are underlinked

Provide test case
KhaledR57
MDEV-37072: Implement IS JSON predicate

Add support for the SQL standard IS JSON predicate with the syntax:
expr IS [ NOT ] JSON [ { VALUE | ARRAY | OBJECT | SCALAR } ]
[ { WITH | WITHOUT } UNIQUE [ KEYS ] ]

The predicate allows checking if an expression is valid JSON
and optionally constrains the JSON type (VALUE, ARRAY, OBJECT,
SCALAR) and whether object keys are unique.

The implementation includes:
- Basic IS JSON validation
- Support for NOT operator
- Type constraints (VALUE, ARRAY, OBJECT, SCALAR)
- Unique keys constraint (WITH/WITHOUT UNIQUE KEYS)
Yuchen Pei
MDEV-37330 [to-squash] Support reverse index
Oleksandr Byelkin
10.6 adjasts
Sergei Golubchik
Merge branch '10.6' into 10.11
Marko Mäkelä
MDEV-23298 fixup: have_perfschema.inc
ParadoxV5
Fix line count accidwentally swapped with pre-5.6

Miraculously, **this did not cause *any* test to consistently fail,
except for an MTR internal check difference from
`rpl.rpl_upgrade_master_info_section`,
from how the server `DEFAULT` is different from the MTR default.
ParadoxV5
MDEV-28302 configurable defaults for CHANGE MASTER

Many CHANGE MASTER fields typically have the same
configurations between multi-source connections – namely:
master_connect_retry
master_ssl
master_ssl_ca
master_ssl_capath
master_ssl_cert
master_ssl_cipher
master_ssl_key
master_ssl_verify_server_cert
master_ssl_crl
master_ssl_crlpath
master_use_gtid
master_retry_count
master_heartbeat_period

When MDEV-25674 added `master_retry_count` to CHANGE MASTER, it kept
the server option `--master-retry-count` to be its default value.
This commit back-adds corresponding server options
for the defaults of the rest of those fields.
With them, the command line or config files can set up common
configurations across replication sources (and even replicas).
`--autoset-master-use-gtid` and `--autoset-master-heartbeat-period` can
also reset their prior corresponding options back to their unset states.

CHANGE MASTER can override unset
(defaulted) configurations per connection.
This commit also adds `DEFAULT` keyword support for all of those fields,
so overridden configurations can reset
to the default without RESET REPLICA.

Supporting passing the `DEFAULT` keyword also enables setting
`master_connect_retry` and `master_retry_count` to 0,
which was previously disregarded.
While here, the commit also increases `master_retry_count`
to 64-bit on LLP64 (e.g., Windows) to match LP64 (e.g., Linux).

Reviewed-by: Brandon Nesterenko <[email protected]>
Reviewed-by: Andrei Elkin <[email protected]>
Yuchen Pei
MDEV-9826 More hash algorithms for PARTITION BY [LINEAR] KEY

PARTITION BY [LINEAR] KEY ALGORITHM={MYSQL51|MYSQL55|BASE31|CRC32C|XXH32|XXH3}

- The MYSQL5X algorithms are the existing algorithms, with MYSQL55
  being the default
- The BASE31 algorithm uses a base-31 representation of the bytes, see
  Modular hashing in https://algs4.cs.princeton.edu/34hash/. It serves
  as a simple baseline that distributes better than the old mysql5x
  algorithms
- CRC32C uses my_crc32c.
- XXH32 and XXH3 are xxhash algorithms - xxhash.h copied from latest
  release (0.8.3) of https://github.com/Cyan4973/xxHash

For performance (esp. xxh) we use one-shot hash functions in binary
hash_sort, and streaming hash function otherwise for byte-by-byte
hashing. XXH is the only stateful hash function. The other hash
algorithms are stateless and homomorphic, so streaming and one-shot
functions are identical, which is reflected in the logic of fallback
from NULL m_hash_byte to m_hash_str applied to a byte.

Tested with

mtr --suite main --do-test=.*partition
mtr --suite parts

Also ran above tests with the following patch that changes the default
algorithm from MYSQL55 to CRC32C, and XXH32 (changing the patch
accordingly)

modified  sql/ha_partition.cc
@@ -10336,6 +10336,8 @@ uint32 ha_partition::calculate_key_hash_value(Field **field_array)
  switch ((*field_array)->table->part_info->key_algorithm)
  {
  case partition_info::KEY_ALGORITHM_NONE:
+    hasher.set_algorithm(HASH_ALGORITHM_CRC32C);
+    break;
  case partition_info::KEY_ALGORITHM_55:
    /* Hasher default to mysql55 */
    break;
modified  sql/partition_info.cc
@@ -2328,7 +2328,7 @@ bool partition_info::fix_parser_data(THD *thd)
      if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
            thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
          key_algorithm == KEY_ALGORITHM_NONE)
-        key_algorithm= KEY_ALGORITHM_55;
+        key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
    }
    DBUG_RETURN(FALSE);
  }
@@ -2344,7 +2344,7 @@ bool partition_info::fix_parser_data(THD *thd)
    if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
          thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
        key_algorithm == KEY_ALGORITHM_NONE)
-      key_algorithm= KEY_ALGORITHM_55;
+      key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
  }
  defined_max_value= FALSE; // in case it already set (CREATE TABLE LIKE)
  do
modified  sql/partition_info.h
@@ -446,6 +446,8 @@ class partition_info : public DDL_LOG_STATE, public Sql_alloc
  int gen_part_type(THD *thd, String *str) const;
};

+#define PARTITION_INFO_DEFAULT_ALGORITHM partition_info::KEY_ALGORITHM_CRC32C
+
void part_type_error(THD *thd, partition_info *work_part_info,
                      const char *part_type, partition_info *tab_part_info);

modified  sql/sql_partition.cc
@@ -2471,7 +2471,7 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
  err+= str->append(STRING_WITH_LEN("KEY "));

  if (part_info->key_algorithm != partition_info::KEY_ALGORITHM_NONE &&
-      part_info->key_algorithm != partition_info::KEY_ALGORITHM_55)
+      part_info->key_algorithm != PARTITION_INFO_DEFAULT_ALGORITHM)
  {
    err+= str->append(STRING_WITH_LEN("ALGORITHM = "));
    switch (part_info->key_algorithm)
@@ -2479,6 +2479,9 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
      case partition_info::KEY_ALGORITHM_51:
        err+= str->append(STRING_WITH_LEN("MYSQL51"));
        break;
+      case partition_info::KEY_ALGORITHM_55:
+        err+= str->append(STRING_WITH_LEN("MYSQL55"));
+        break;
      case partition_info::KEY_ALGORITHM_BASE31:
        err+= str->append(STRING_WITH_LEN("BASE31"));
        break;
Aleksey Midenkov
MDEV-32317 ref_ptrs exhaust on multiple ORDER by func from winfunc

Each ORDER and WHERE slot may generate split, see code like this:

  if ((item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM) ||
    item->with_window_func())
  item->split_sum_func(thd, ref_ptrs, all_fields, SPLIT_SUM_SELECT);

Such kind of code is done in JOIN::prepare(), setup_order(),
setup_fields(), setup_group() and split_sum_func2() itself.

Since we are at the phase of ref_ptrs allocation, items are not fixed
yet and we cannot calculate precisely how much ref_ptrs is needed. We
can estimate at most how much is needed. In the worst case each window
function generates split on each ORDER BY field, GROUP BY field and
WHERE field, so the counts of these should be multiplied by window
funcs count.

As the split can be done in both setup_without_group() and
JOIN::prepare() simultaneously, the factor of window funcs should be
multiplied by 2.

The similar case may be with inner sumfunc items as of the condition

  item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM

but factor of these is harder to predict at the stage of unfixed
items.
ParadoxV5
MDEV-37530: Use boolean OR, not bitwise OR

Unlike Boolean operators,
the order of operands of bit-logic operators is not guaranteed.

`DBUG_ASSERT()` uses the boolean value and
does not show the integer value anyway.

Co-authored-by: Brandon Nesterenko <[email protected]>
KhaledR57
MDEV-36107 MDEV-36108 Enhance mysqltest language with expression evaluation and variable substitutions

mysqltest had limited scripting capabilities, requiring complex
workarounds for mathematical calculations and string manipulations
in test cases. This commit solves these limitations by adding a new
`$(...)` syntax that enables direct evaluation of mathematical, logical,
and string expressions within test scripts.

Expression Evaluation (MDEV-36107):
- Recursive descent parser supporting arithmetic, logical, comparison,
  and bitwise operators with proper precedence
- Support for integers (decimal, hex, binary), booleans, strings, and
  NULL values
- Variable substitution within expressions
- Integration with existing mysqltest control flow

String Functions (MDEV-36108):
- Base conversion functions supporting bases 2-62
- String manipulation and processing functions
- Regular expression functions
- Conditional and numeric utility functions

The implementation enhances mysqltest's scripting capabilities while
maintaining full backward compatibility.
ParadoxV5
MDEV-37530 fixes

* MDEV-38410: Use array, not `std::initializer_list`
  Some environments appear not to retain the backing array of a
  static `std::initializer_list` in the MDEV-37530 release candidate,
  and eventually crash when reading overwritten data.
  This commit resolves the stealth issue by reverting to conventional
  arrays, while maintaining convenience through deductive overloads.
* Compile problems
  * Some of our platforms (namely SUSE 15, which uses GCC 7.5) support
    C++17 syntaxes, but not all libraries, `<charconv>`` among those.
* Update to the current `main` branch

Co-authored-by: Sergei Golubchik <[email protected]>
Co-authored-by: Brandon Nesterenko <[email protected]>
gkodinov
MDEV-38642: Missing Null terminator in the definition of mysqldump's --system typelib

There was a missing NULL element terminator for --system's type
library definition.

This was causing a crash in find_type_eol when e.g. an incomplete
value was passed to --system where it keeps iterating until it
finds the NULL as a typelib element.

Fixed by appending a NullS to the definition.
Test case added.
ParadoxV5
MDEV-38410: Use array, not `std::initializer_list`

Some environments appear not to retain the backing array of a
static `std::initializer_list` in the MDEV-37530 release candidate,
and eventually crash when reading overwritten data.

This commit resolves the stealth issue by reverting to conventional
arrays, while maintaining convenience through deductive overloads.
Yuchen Pei
MDEV-9826 [to-squash] Fix view protocol test

view protocol uses a separate connection so we have to set global
debug_dbug too
ParadoxV5
MDEV-37530 Refactor Master & Relay Log info to iterable tuples

The persistence code of CHANGE MASTER values needs to match file
lines with the formatting for the corresponding field’s type.
This is unstructured, repetitive (not DRY), and makes feature expansions
(such as MDEV-28302 `CHANGE MASTER …=DEFAULT`)
error-prone if not difficult.

This commit moves these functions and global constants from
`slave.cc`/`.h`, as well as the Master and Relay Log Info File
entries from `Rpl_mi` and `Rpl_rli`, to dedicated `rpl_*info_file.h`
files and under corresponding structs to facilitate organization.
Namely, this commit wraps those file entries with transparent
structs that inherit from a `Persistent` interface,
or shared helper structs that themselves inherit from `Persistent`.
By moving the file read/write helper functions to or behind
implementations of `Persistent`’s virtual methods,
reading or writing the file only takes a type-agnostic loop
over the (wrapped) CHANGE MASTER values.

* This commit also includes preemptive support
  for preserving MDEV-28302’s `=DEFAULT`.
  As such, unset fields (namely `master_connect_retry`)
  now remember their `DEFAULT` states
  rather than whatever the default is at CHANGE MASTER time.
* For consistency’s sake, `master_heartbeat_period` is
  now reset at RESET REPLICA instead of CHANGE MASTER.

As this refactor will disconnect it from fixes for
some open bugs in prior versions, this commit also:
* Reimplements the value reader functions to be strict with their input
  * Fixes MDEV-38010
    number parsing ignores trailing garbage and overflows
  * Supercedes MDEV-38020 integer overflow
* Changes master_heartbeat_period from a `float` of seconds
  to a `uint32_t` of milliseconds (adding `/1000.0`s as needed)
  * Fixes MDEV-35879 `Slave_heartbeat_period` is imprecise
  * The maximum of `master_heartbeat_period` has been
    increased to 4294967.295, i.e., (2³²-1)÷1000.
  * `master_heartbeat_period` now rounds
    instead of truncates (rounds down).
* Prepares to make `master_retry_count` 64-bit on
  LLP64 (e.g., Windows) to match LP64 (e.g., Linux)

Reviewed-by: Brandon Nesterenko <[email protected]>
Thirunarayanan Balathandayuthapani
MDEV-38667  Assertion in diagnostics area on DDL stats timeout

Reason:
======
During InnoDB DDL, statistics updation fails due to lock wait
timeout and calls push_warning_printf() to generate warnings
but then returns success, causing the SQL layer
to attempt calling set_ok_status() when the diagnostics area
is already set.

Solution:
=========
By temporarily setting abort_on_warning to false around operations
that prevents warning to error escalation and restore the original
setting after calling HA_EXTRA_END_ALTER_COPY for alter operation.
ParadoxV5
`decimal_out` `sometimes-uninitialized`

Be defensive: Do not assign out if any conversion step fails.
Sergei Golubchik
Merge branch '10.11' into 11.4
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read ahead had better be logical

- Ported MDEV-32067 branch into main. Added a few interface
for multi-range read limit and InnoDB also aware of number
of pages to be read. Yet to integrate with buf_read_ahead_pages()
Thirunarayanan Balathandayuthapani
MDEV-32067 InnoDB linear read ahead had better be logical

- Ported MDEV-32067 branch into main. Added a few interface
for multi-range read limit and InnoDB also aware of number
of pages to be read. Yet to integrate with buf_read_ahead_pages()
ParadoxV5
MDEV-37530: Avoid C++17’s `<charconv>` entirely

* Some of our platforms (namely SUSE 15, which uses GCC 7.5)
  support C++17 syntaxes, but not all libraries, <charconv> among those.
* We are yet to figure out what to do with our 10 choices of string→int
  converters, so now is not the time to settle preferences on <charconv>
  or whatever superset that covers all of their cases.

Co-authored-by: Sergei Golubchik <[email protected]>
Yuchen Pei
MDEV-37330 [to-squash] Address github review input

- basic implementation of bitmap_get_prev_set
- a testcase to improve coverage
- handle multiple PARTITION BY RANGE COLUMNS in Case 1, with the
  limitation that all keys should have the same direction
Sergei Golubchik
Merge branch '10.11' into 11.4
Sergei Golubchik
MDEV-38604 fix SP execution too
Yuchen Pei
MDEV-38522 Reset m_pi_scan_method in ha_partition::reset
Sergei Golubchik
Merge branch '11.4' into 11.8
Sergei Golubchik
MDEV-32317 fix the test for --view
ParadoxV5
MDEV-37530 warning: make `int`→`uchar` explicit

Arithmetics start from `int` and so auto-promotes.