Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Alexey Yurchenko
MDEV-40179 Found N prepared transactions after mariabackup SST

With log_bin=ON a transaction is committed via two-phase commit (the
binary log is the second participant), so it passes through the InnoDB
XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup
backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more
such writesets prepared-but-not-yet-committed, and the snapshot captures
them. On a freshly SST'd joiner nothing resolves these prepared
transactions: binlog crash recovery does not run (the joiner has no in-use
binlog to recover from), and the wsrep continuity-based commit is inactive
because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover
prepared transactions then abort startup with "Found <N> prepared
transactions!". Note this does not depend on the prepared set being
non-contiguous - even a contiguous run aborts, because nothing commits
or rolls it back.

Rollback these transactions in xarecover_handlerton().

Co-Authored-By: Claude Opus 4.8 <[email protected]>
rusher
[misc] Add matrix entry that builds against latest C/C 3.3 branch
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
Made prepearedstatement testsuite not to force CSPS for defautl connection

This way it can be controlled via params/env which typ of PS is tested.
Tests requiring CSPS should take care of that
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
Removed polymorphysm from the row classes

Polymorfism is done my their respective resultset host class - text RS
always use text row class, binary RS - binary row. Thus no need to
access their features via virtual methods.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
CONCPP-157, CONCPP-158 setBytes() parameter is sent as binary type in

binary protocol. It is sent as MYSQL_TYPE_BLOB. As counterpart, binary
types received from the server via binary protocol are now converted
to int types - berfore the exception would be thrown right away w/out
any attempt to convert.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
drrtuy
chore(privs): restrict access to run_in_duckdb() to SUPER users only.
drrtuy
chore: enable run_in_duckdb by default()
Alexey Yurchenko
MDEV-38147 error 1950 after mariabackup SST with gtid_strict_mode=ON

After a mariabackup SST the joiner could fail with

  ER_GTID_STRICT_OUT_OF_ORDER (error 1950)

while re-binlogging transactions received over IST.

The cause is that the binary log copied from the donor carries a
Gtid_list whose position can be ahead of the storage-engine snapshot:
BACKUP STAGE BLOCK_COMMIT blocks the engine commit (2PC step 3) but not
the binary log write (step 2), so transactions can be present in the
copied binlog that are not committed in the copied engine snapshot.
After the SST the joiner reports the (committed) engine position to the
cluster, IST resends those transactions, and re-binlogging them under
gtid_strict_mode=ON collides with the ahead Gtid_list -> error 1950.
(MDEV-34483 made the engine snapshot stop short of the binlog, which is
what exposed this.)

The copied binary log carries no transactions the joiner needs - only a
Gtid_list - so instead of shipping and then having to truncate/reconcile
it, the joiner now starts a fresh binary log and seeds its GTID position
from the storage-engine checkpoint during recovery. That checkpoint is
the committed cluster position, i.e. exactly where IST resumes, so the
joiner's binary log stays in lockstep with the rest of the cluster and
no out-of-order GTID can occur.

This works for both wsrep_gtid_mode settings; only the binlog domain of
the cluster stream differs:

  - wsrep_gtid_mode=ON : wsrep_gtid_domain_id (cluster writes are
    re-tagged to it), which is the domain stored in the checkpoint;
  - wsrep_gtid_mode=OFF: gtid_domain_id (cluster writes keep the node's
    configured domain).

Async-replica positions (mysql.gtid_slave_pos) are part of the engine
snapshot and survive the SST unchanged, so a Galera node can still serve
as an async master or replica across the SST.

This commit:
- sql/log.cc: adds wsrep_seed_binlog_gtid_state(), called from
  do_binlog_recovery() when the joiner has no binary log, seeding the
  binlog GTID state for the cluster domain to the SE checkpoint position.
- scripts/wsrep_sst_mariabackup.sh: no longer moves the donor's binary
  log into place on the joiner.
- extra/mariabackup: backward compatibility: keep shipping binlog file
  in SST but
  - on donor fix the race between rotation and shipping so that the file
    shipped is the one that had been rotated;
  - on joiner discard shipped binlog in favour of one generated by
    wsrep_seed_binlog_gtid_state().
- sql/wsrep_sst.cc: logs the position actually adopted from storage
  (the authoritative post-SST position) rather than the script-reported
  one.
- sql/handler.cc: downgrades the "Discovered discontinuity in recovered
  wsrep transaction XIDs" message in wsrep_order_and_check_continuity()
  from warning to debug level. With parallel appliers a snapshot
  routinely captures prepared XIDs that are not contiguous with the
  engine checkpoint, so this is normal during SST recovery and of no
  value in regular operation; the transactions past the checkpoint are
  re-delivered by the cluster (IST/SST) regardless.
- Adds an MDEV-38147 MTR test reproducing the issue.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Lawrin Novitsky
Moved underlying libmariadb to 3.4 series - v3.4.9 atm

We have to support parsec auth plugin as minimum. zero config TLS is
also a big plus, but we don't change related options defaults - the
certificate is not yet validated by default.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
CONCPP-153 Parameter value escaping did not consider the charset

In case of client side statement prepare, if the data is encoded in multibyte
charsets, escaping of string and binary parameter could lead to incorrect data
that in its turn could let sql injection with charsets like big5, gbk, sjis, cp932.

The fix considers the charset now. The testcase has been added.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Rex
MDEV-35565 Server crashes simplifying group by <subquery>

During optimization, we may call update_depend_map_for_order()
on a group by list containing an expression with an outer reference.
The table map for this outer reference will be collected by
Item_subselect::recalc_used_tables and used to return a depend_map.
This map will indicate the presence of tables not in this select and
can cause erroneous results, poor execution plans or crashes.
We update the class used to collect these bitmaps to check whether the
containing select_lex is applicable, and if not, set the OUTER_REF_TABLE_BIT
bit in the table map.

Approved-by: Dave Gosselin ([email protected])
Lawrin Novitsky
Merge branch 'cpp-1.0' into cpp-1.1
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
drrtuy
chore(tests): add data reclaim test.
Lawrin Novitsky
Introduced UBSAN.supp to supress UBSAN issues beyond our control

(on C/C side). They are quite the same as in C/ODBC
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
Minor formatting changes
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
Removed polymorphysm from the row classes

Polymorfism is done my their respective resultset host class - text RS
always use text row class, binary RS - binary row. Thus no need to
access their features via virtual methods.
drrtuy
chore(tests,docs): renew documentation and fixed a number of test inconsistences.
rusher
[misc] add maxscale testing to CI
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
Fix of UBSAN errors.

In one place copying from nullptr was possible. It could happen if the
streamed text resultset had to be cached(if the connection is requested
for other operation), and NULL value was present.
In couple of places int64_t could be overflown - had to be changed to
uint64 as it is defined plus othe implied changes.
In particular fetching MIN_INT64 value in text protocol could cause
that. Looks like all compilers work fine in this case as there is the
test for this.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
One more UBSAN silencing.

Same nature "function called using wrong pointer type" for C/C parameters
callback call. In this case besides inclusion in the namespace the
function had accidentally wrong return type.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
CONCPP-152 fix of effect of C/C resetting length of fixed size types

The driver was using that length directly. Now it tests the length and
use either it or buffer length.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Razvan Liviu Varzaru
Support fedora for static_test linking
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
rusher
[misc] add maxscale testing to CI
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
drrtuy
chore: default memory limits are now 80% * (physical RAM - innodb_buffer_pool_size_max)
Lawrin Novitsky
Merge branch 'cpp-1.0' into cpp-1.1
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
BinRow class as value in binary resultSet class
Rex
MDEV-35565 Server crashes simplifying group by <subquery>

During optimization, we may call update_depend_map_for_order()
on a group by list containing an expression with an outer reference.
The table map for this outer reference will be collected by
Item_subselect::recalc_used_tables and used to return a depend_map.
This map will indicate the presence of tables not in this select and
can cause erroneous results, poor execution plans or crashes.
We update the class used to collect these bitmaps to check whether the
containing select_lex is applicable, and if not, set the OUTER_REF_TABLE_BIT
bit in the table map.

Approved-by: Dave Gosselin ([email protected])
Lawrin Novitsky
CONCPP-153 Parameter value escaping did not consider the charset

In case of client side statement prepare, if the data is encoded in multibyte
charsets, escaping of string and binary parameter could lead to incorrect data
that in its turn could let sql injection with charsets like big5, gbk, sjis, cp932.

The fix considers the charset now. The testcase has been added.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Alexey Yurchenko
MDEV-40179 Found N prepared transactions after mariabackup SST

With log_bin=ON a transaction is committed via two-phase commit (the
binary log is the second participant), so it passes through the InnoDB
XA-prepare state. While a donor is held in BLOCK_COMMIT for a mariabackup
backup, its parallel appliers (wsrep_slave_threads > 1) leave one or more
such writesets prepared-but-not-yet-committed, and the snapshot captures
them. On a freshly SST'd joiner nothing resolves these prepared
transactions: binlog crash recovery does not run (the joiner has no in-use
binlog to recover from), and the wsrep continuity-based commit is inactive
because wsrep_emulate_bin_log is FALSE when log_bin is ON. The leftover
prepared transactions then abort startup with "Found <N> prepared
transactions!". Note this does not depend on the prepared set being
non-contiguous - even a contiguous run aborts, because nothing commits
or rolls it back.

Rollback these transactions in xarecover_handlerton().

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Lawrin Novitsky
Merge branch 'cpp-1.1'
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
drrtuy
chore: bump DuckDB version to 1.5.4.
Alexey Yurchenko
MDEV-38147 error 1950 after mariabackup SST with gtid_strict_mode=ON

After a mariabackup SST the joiner could fail with

  ER_GTID_STRICT_OUT_OF_ORDER (error 1950)

while re-binlogging transactions received over IST.

The cause is that the binary log copied from the donor carries a
Gtid_list whose position can be ahead of the storage-engine snapshot:
BACKUP STAGE BLOCK_COMMIT blocks the engine commit (2PC step 3) but not
the binary log write (step 2), so transactions can be present in the
copied binlog that are not committed in the copied engine snapshot.
After the SST the joiner reports the (committed) engine position to the
cluster, IST resends those transactions, and re-binlogging them under
gtid_strict_mode=ON collides with the ahead Gtid_list -> error 1950.
(MDEV-34483 made the engine snapshot stop short of the binlog, which is
what exposed this.)

The copied binary log carries no transactions the joiner needs - only a
Gtid_list - so instead of shipping and then having to truncate/reconcile
it, the joiner now starts a fresh binary log and seeds its GTID position
from the storage-engine checkpoint during recovery. That checkpoint is
the committed cluster position, i.e. exactly where IST resumes, so the
joiner's binary log stays in lockstep with the rest of the cluster and
no out-of-order GTID can occur.

This works for both wsrep_gtid_mode settings; only the binlog domain of
the cluster stream differs:

  - wsrep_gtid_mode=ON : wsrep_gtid_domain_id (cluster writes are
    re-tagged to it), which is the domain stored in the checkpoint;
  - wsrep_gtid_mode=OFF: gtid_domain_id (cluster writes keep the node's
    configured domain).

Async-replica positions (mysql.gtid_slave_pos) are part of the engine
snapshot and survive the SST unchanged, so a Galera node can still serve
as an async master or replica across the SST.

This commit:
- sql/log.cc: adds wsrep_seed_binlog_gtid_state(), called from
  do_binlog_recovery() when the joiner has no binary log, seeding the
  binlog GTID state for the cluster domain to the SE checkpoint position.
- scripts/wsrep_sst_mariabackup.sh: no longer moves the donor's binary
  log into place on the joiner.
- extra/mariabackup: backward compatibility: keep shipping binlog file
  in SST but
  - on donor fix the race between rotation and shipping so that the file
    shipped is the one that had been rotated;
  - on joiner discard shipped binlog in favour of one generated by
    wsrep_seed_binlog_gtid_state().
- sql/wsrep_sst.cc: logs the position actually adopted from storage
  (the authoritative post-SST position) rather than the script-reported
  one.
- sql/handler.cc: downgrades the "Discovered discontinuity in recovered
  wsrep transaction XIDs" message in wsrep_order_and_check_continuity()
  from warning to debug level. With parallel appliers a snapshot
  routinely captures prepared XIDs that are not contiguous with the
  engine checkpoint, so this is normal during SST recovery and of no
  value in regular operation; the transactions past the checkpoint are
  re-delivered by the cluster (IST/SST) regardless.
- Adds an MDEV-38147 MTR test reproducing the issue.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
drrtuy
MDEV-40265 fix: UUID type support in DuckDB and mixed SELECT statements.
Lawrin Novitsky
Merge branch 'cpp-1.0' into cpp-1.1
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Lawrin Novitsky
CONCPP-156 Move to C/C 3.4 to support Parsec plugin

The submodule has been moved to 3.4.9 already in one of previous
commits. This one makes it it installed by MSI and taraballs
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
drrtuy
chore(build): suppress sfinae-incomplete warnings building DuckDB code to reduce noise for >=] GCC-16 compiler.
drrtuy
fix(func): time functions fixes and tests.
Rex
MDEV-35565 Server crashes simplifying group by <subquery>

During optimization, we may call update_depend_map_for_order()
on a group by list containing an expression with an outer reference.
The table map for this outer reference will be collected by
Item_subselect::recalc_used_tables and used to return a depend_map.
This map will indicate the presence of tables not in this select and
can cause erroneous results, poor execution plans or crashes.
We update the class used to collect these bitmaps to check whether the
containing select_lex is applicable, and if not, set the OUTER_REF_TABLE_BIT
bit in the table map.

Approved-by: Dave Gosselin ([email protected])
Lawrin Novitsky
More fixes to please UBSAN and MSAN.

For UBSAN it's the similar problem as in prvious commit, for MSAN - it
suddenly had issues afte move of min required cmake version to 3.5.
Seemingly it is connected to policy 0056 and it's new behavior.
  • ccpp-sles12-amd64: build linux-connector_cpp failed -  stdio
Sergei Golubchik
github copilot review instructions

Co-Authored-By: Claude Opus 4.8 <[email protected]>