Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
MDEV-39673 group_concat ignores max_allowed_packet

GROUP_CONCAT is limited by group_concat_max_len,
but also, as a string function it must respect max_allowed_packet.

Let's introduce THD::gconcat_max_len() helper to simplify checks.

Also:
* make max group_concat_max_len value the same as max max_allowed_packet
* use the same MY_MIN((ulonglong) ..., UINT_MAX32) in
  Item_func_json_objectagg as in Item_func_group_concat
* use overflow-safe type for lengths in Item_func_quote
  (MAX_MAX_ALLOWED_PACKET is 1G so uint cannot overflow yet, but it's
  a fragile assumption)
Vladislav Vaintroub
Workaround OOM errors during test on Windows ARM64 github actions runner.

Run less parallel workers. This did not slow the build, so disk IO seems
to be the limiting factor on this machine, not the CPU.

Also do not run replication.
Jan Lindström
MDEV-39685 : galera multi table update crash

This is regression caused by MDEV-28750 commit 1f349968.

In multi-table update table list could contain tables
that are not yet opened because update does not really
change them. This can happen e.g. when update changes
table that is referenced by foreign key by table that
is not part of multi-table update.

Fixed by first checking is wsrep write set size limited.
If it is not multi-table update can continue normally.
If write set size is limited then check has update
updated both transactional and non-transactional tables
and those tables that have not yet been opened can be
safely skipped as they are not updated.
Jan Lindström
Fix merge error on test case.
Sergei Golubchik
Revert "MDEV-17677: Keywords followed by .number parsed as identifiers"

This reverts commit 895b28d6721eadfad0d47723fcc949eae75cf8cf.
Hemant Dangi
MDEV-39648: wsrep_sst_rsync.sh: apply safe() to joiner-supplied parameters

Issue:
wsrep_sst_rsync.sh interpolated WSREP_SST_OPT_REMOTE_USER and
WSREP_SST_OPT_REMOTE_PSWD verbatim. Because both values originate from
the joiner side of the SST request, a newline in either could splice
an extra directive into the donor-written stunnel.conf (silently
downgrading peer-cert verification) or an extra line into the rsync
magic file. MDEV-39413 had introduced safe() for the same threat class
in wsrep_sst_mariabackup but did not extend it to the rsync script.

Solution:
Routing the rsync interpolations through safe() closes the gap, and
extending safe() to also reject tab and newline ensures multi-line
values cannot survive into a config-file heredoc.
Rex Johnston
MDEV-39492 Parallel Query: Study how to create worker threads

Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.

2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name).  Main management
class created in the stack in JOIN::exec, implemented for the
top level select.

Current parallel_worker_thread_func sleeps for 10 seconds, generates
a warning, signals the main thread, sleeps 10 seconds, signals the
main thread again, sets it's finished flag and cleans it's THD.

The main thread loops through worker threads, looking for finished
thread and cleans them up if they have finished.
It then waits for a signal, then processes it's message queue.

The thread management data is allocated on the stack in JOIN::exec.
Everything else is allocated using my_malloc() and my_free().

Threads are registed in server_threads, so are visible in
information_schema.processlist and the show processlist command.

We check that a kill query on a parallel worker is passed onto it's
manager and the query is properly aborted, and that a kill connection
is handled properly in parallel_worker.test.

Review input 1: cleanup earlier

Do cleanup before we've finished sending the result to the client.
This way, one can see the errors (and eventually warnings) marshalled
back to the main thread and returned to the user:

MariaDB [test]> set parallel_worker_threads=10;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> select seq from seq_1_to_10;
ERROR 4103 (HY000): Argument to the worker_busted_function() function does not belong to the range [0,1]
Sergei Golubchik
MDEV-39564 One-byte OOB write in PROXY protocol v1 header parser
bsrikanth-mariadb
MDEV-39440: Failed to match the stats from replay context with the optimizer stats

The problem is that handler->multi_range_read_info_const() call in
check_quick_select() returned rows equal to HA_POS_ERROR, both during
context capture and replay. However, we never stored the ranges info
into the context when rows=HA_POS_ERROR. However, during replay, we try
to infuse stats for the given range, and since no match was found in the
context, we produced a warning.

Solution is to store range_info for all the ranges even when they get
HA_POS_ERROR number of rows.
Rex Johnston
MDEV-39492 Parallel Query: Study how to create worker threads

Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.

2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name).  Main management
class created in the stack in JOIN::exec, implemented for the
top level select.

Current parallel_worker_thread_func sleeps for 10 seconds, generates
a warning, signals the main thread, sleeps 10 seconds, signals the
main thread again, sets it's finished flag and cleans it's THD.

The main thread loops through worker threads, looking for finished
thread and cleans them up if they have finished.
It then waits for a signal, then processes it's message queue.

The thread management data is allocated on the stack in JOIN::exec.
Everything else is allocated using my_malloc() and my_free().

Threads are registed in server_threads, so are visible in
information_schema.processlist and the show processlist command.

We check that a kill query on a parallel worker is passed onto it's
manager and the query is properly aborted, and that a kill connection
is handled properly in parallel_worker.test.

Review input 1: cleanup earlier

Do cleanup before we've finished sending the result to the client.
This way, one can see the errors (and eventually warnings) marshalled
back to the main thread and returned to the user:

MariaDB [test]> set parallel_worker_threads=10;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> select seq from seq_1_to_10;
ERROR 4103 (HY000): Argument to the worker_busted_function() function does not belong to the range [0,1]
Sergei Golubchik
MDEV-39676 disallow global.wsrep_sst_donor=NULL again

it crashes in galera.mdev-28433
Sergei Golubchik
Merge branch '11.4' into 11.8
sjaakola
MDEV-34784 unhandled FK dependency with DML vs DDL

Certain DDL statements (e.g. ALTER TABLE) require innodb table lock
on tables having foreign key constraint reference to the table
under DDL execution. This dependency is not added in write set
key information. However, tables being referenced to will be added
in the key information, so the table locking domain of DDL is only
partially recorded.

One harmful consequence of this missing dependency information happens
when a DML modifies a FK child table's row, which has NULL in the FK
referencing column. In such situation, the FK reference cannot be followed
during DDL execution, and there will be no FK parent table keys recorded
in the write set. Parallel applying (or multi-master access) of such DML
and DDL on the FK parent table will cause applying conflicts.

This  scenario is presented in a new mtr test added in this commit.
The commit has a fix for the DDL FK dependency handling by adding all FK
child table names in the write set key information.
The commit has also fixes for innodb lock0lock.cc error logging to report
lock connflicts of table and record locks correctly.
Sergei Golubchik
MDEV-39581 dynamic column header missing sanity checks
Dave Gosselin
FULL JOIN, straight_join and optimizer hint support

Previously the FULL JOIN tables had to come first in the join
order, before any other table.  They still have to stay next to
each other, but they can now appear before, after, or between
other tables in the FROM clause.

Update the JOIN_ORDER, JOIN_PREFIX, and JOIN_SUFFIX hint conflict
check to match.  The new check only rejects a hint when it would force
a non-FULL JOIN table to sit between two FULL JOIN tables, which is
the case that actually breaks the null-complement rescan.  It finds
this by computing the set of predecessors of FULL JOIN tables and the
set of successors of FULL JOIN tables, after the hint's dependencies
have been added, and rejects the hint when those sets overlap on a
non-FULL-JOIN, non constant table.
Daniel Bartholomew
bump the VERSION
Rex Johnston
MDEV-39492 Parallel Query: Study how to create worker threads

Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.

2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name).  Main management
class created in the stack in JOIN::exec, implemented for the
top level select.

Current parallel_worker_thread_func sleeps for 10 seconds, generates
a warning, signals the main thread, sleeps 10 seconds, signals the
main thread again, sets it's finished flag and cleans it's THD.

The main thread loops through worker threads, looking for finished
thread and cleans them up if they have finished.
It then waits for a signal, then processes it's message queue.

The thread management data is allocated on the stack in JOIN::exec.
Everything else is allocated using my_malloc() and my_free().

Threads are registed in server_threads, so are visible in
information_schema.processlist and the show processlist command.

We check that a kill query on a parallel worker is passed onto it's
manager and the query is properly aborted, and that a kill connection
is handled properly in parallel_worker.test.

Review input 1: cleanup earlier

Do cleanup before we've finished sending the result to the client.
This way, one can see the errors (and eventually warnings) marshalled
back to the main thread and returned to the user:

MariaDB [test]> set parallel_worker_threads=10;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> select seq from seq_1_to_10;
ERROR 4103 (HY000): Argument to the worker_busted_function() function does not belong to the range [0,1]
Daniel Black
MDEV-39675 mariadb_upgrade utility failed when path=nonexist

The upgrade SQL assumes the functions will resolve.

With a non-default path this isn't true and results in:

ERROR 1305 (42000) at line 3211: FUNCTION extract_schema_from_file_name does not exist

Resolve by setting PATH = 'CURRENT_SCHEMA' for the upgrade.

Idea thanks to Sergei Golubchik
Marko Mäkelä
MDEV-13542 fixup: Remove orphan trx_print()
Sergei Golubchik
fix columnstore for new cmake
Sergei Golubchik
MDEV-39658 ASAN crash on invalid proxy_protocol_networks value

max_subnet estimate was off by one.

* only allow address with a known family
  (trips an assert in addr_matches_subnet())
* adjust max_subnet estimate accordinly
Sergei Golubchik
Merge branch '10.11' into 11.4
Dave Gosselin
Make FULL JOIN work correctly when other tables sit beside it

A FULL JOIN produces its result in two stages.  The first stage walks
the joined tables like a LEFT JOIN, emitting every left row paired
with its matches on the right (or with NULLs if there were none).  The
second stage walks the right table again and emits the right rows that
never found a match, padded with NULLs on the left.

Until now the second stage ran exactly once per query, after the whole
first stage had finished.  That worked only when the FULL JOIN was the
entire query.  As soon as another table sat outside the FULL JOIN, the
result became wrong, because the "right rows with no matches" needed
to be paired with the outer table's rows just like the matched rows
were.  To avoid producing wrong answers, the optimizer was forced to
put FULL JOIN tables first.

This change moves the second stage so that it runs as part of the same
nested loop that produced the first stage's matches.  Each time the
outer loop advances to a new row, the right table is rescanned for its
unmatched rows, and those rows flow through the rest of the query
exactly the way ordinary join output does.

With the two stages now interleaved, the optimizer no longer has to
force the FULL JOIN to the front of the join order.  Tables outside
the FULL JOIN can sit on either side of it, and the FULL JOIN itself
only needs to stay together as one contiguous group.
Dave Gosselin
MDEV-39663: Assertion `found_tables > 0' failed

Early on in FULL OUTER JOIN development, it seemed prudent to
attach the ON condition to both tables during FULL OUTER JOIN
parsing, because either table in the join could be considered
the inner or outer table, depending on perspective.  This
decision ended up complicating both simplify_joins and the
FULL JOIN rewrites to other join types, as well as dependent
table propagation during make_join_statistics.  Dependent
table propagation was not correct because both tables in the
FULL JOIN carried the ON expression which confused
make_join_statistics.

We could update make_join_statistics to propagate dependent
tables for both sides of a join, but that's essentially changing
the mainstream join processing path for the sake of one case
case.  Instead, just stop attaching the ON condition to both
tables in a FULL OUTER JOIN, which simplifies the FULL JOIN
rewrites and relies on pre-existing table dependency propagation
in make_join_statistics.
Sergei Golubchik
Merge branch '10.6' into 10.11
Vladislav Vaintroub
wip
Vladislav Vaintroub
wip
Jan Lindström
Prepare for Galera library version 26.4.27
Vladislav Vaintroub
CI - Fix Github Actions warning, bump actions/checkout version
Sergei Golubchik
MDEV-39657 ASAN error on malformed WKB point

check data length for Gis_point
Sergei Golubchik
proxy protocol v2: fix a harmless typo

according to the rfc, the length is 2 bytes,
but the max length is 226 and there's a validity
check for length <= 240.
Sergei Golubchik
MDEV-39654 schema-qualified unquoted table name starting with digit fails to parse

add tests
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME
- at least one partition is supplied
- INTERVAL interval is a time interval

When performing DML on such a table, it will first add partitions
by the specified interval until the partition covers the current time.

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.
Vladislav Vaintroub
wip
Yuchen Pei
MDEV-15621 Auto add RANGE COLUMNS partitions by interval

Allow auto partitioning by interval in PARTITION BY RANGE COLUMNS

PARTITION BY RANGE COLUMNS (col_name)
INTERVAL interval [AUTO]
(
  PARTITION partition_name VALUES LESS THAN (value)
  [, PARTITION partition_name VALUES LESS THAN (value) ... ]
)

where

- col_name is the name of one column of type DATE or DATETIME
- at least one partition is supplied
- INTERVAL interval is a time interval

When performing DML on such a table, it will first add partitions
by the specified interval until the partition covers the current time.

Partition addition will not cause an implicit commit like DDL normally
does.

The partitions are named pN.

Otherwise the table behaves exactly the same as a normal RANGE COLUMNS
partitioned table.
Sergei Golubchik
MDEV-39565 missing filename check in mariadb-backup --decompress

check for tablename-safe characters in backed up table files
Jan Lindström
MDEV-39676 : Galera Cluster-peer > Donor command execution

Add verification of wsrep_sst_donor, wsrep_sst_method and
wsrep_sst_receive_address so that they contain only
supported characters. But allow NULL or empty value.
Sergei Golubchik
MDEV-39622 OBJECT_INSTANCE_BEGIN in P_S are unstable, difficult to compare

let's show stable values that don't change between runs
Sergei Golubchik
MDEV-39576 PROXY v2 protocol uninitialized memory reads
Jan Lindström
Fix test failure on galera_sst_mariabackup_encrypt_with_key_server

Joiner mariadbd exits when SST is aborted; the exit code varies by
platform (clean 0 on some systems, signalled 134 / 1 on others).

pkill exit code can also vary by platform (clean 0 on some systems,
signalled 1 others).