Console View

Categories: connectors experimental galera main

Legend:

Passed

Failed

Warnings

Failed Again

Running

Exception

Offline

No data

connectors

experimental

galera

main

fee9458c6b4f...

fee9458c6b4fd3ce833f05ac4c1ae4e3c5813362

Thirunarayanan Balathandayuthapani

thiru@mariadb.com

MDEV-39061 mariadb-backup compatible wrapper for BACKUP SERVER

This adds a shell script that lets users keep using their existing
mariadb-backup commands while the real work is done by the new
server-side BACKUP SERVER command. The goal is "drop-in": users should
not have to change their backup scripts.

extra/mariabackup/scripts/mariadb-backup-server.sh (plain POSIX sh)
understands the usual mariadb-backup modes and translates each one.
A companion helper, extra/mariabackup/scripts/mbstream-server.sh,
lets streamed backups be unpacked by pipelines that expect the
mbstream CLI. Both are documented in
extra/mariabackup/scripts/README.md.

--backup
========
Connects with the mariadb client and runs "BACKUP SERVER TO '<dir>'".
Connection options (--user, --host, --port, --socket, --defaults-file,
ssl, ...) are passed through to the client; --parallel=N becomes the
"<N> CONCURRENT" clause.
After the backup it writes backup-prepare.cnf into the backup
directory, recording what --prepare needs later: where
mariadbd lives, the InnoDB parameters (page size, data file path,
undo tablespaces, checksum algorithm, log file size), and if
the server is encrypted then how to reload the encryption key
plugin (the file_key_management variables),
so an encrypted backup can be prepared without extra input.

--backup --stream
=================
Runs "BACKUP SERVER WITH [N CONCURRENT] '<command>'": the server feeds
each stream's tar to <command>, the wrapper collects the parts, writes
them to stdout, then appends backup-prepare.cnf as a final tar. The
per-stream tars carry no end-of-archive marker; only the trailing
backup-prepare.cnf adds the single end marker, so the whole stream
extracts with a plain "tar -x".
Two properties follow from how BACKUP SERVER streams,
both differing from mariadb-backup:
- local: the stream command runs inside the server, so the wrapper
must share its filesystem;
- tar only: any --stream=<format> (including xbstream) yields tar.
--target-dir is optional in stream mode (scratch for the per-stream
parts; a mktemp dir is used otherwise).

mbstream-server.sh maps the mbstream CLI onto a plain "tar -x"/"tar -c", so
existing "mbstream -x"/"-c" pipelines keep working on the wrapper's
stream. mbstream-only flags (-p/--parallel, ...) are accepted and
ignored; any other unknown option is rejected.

Environment overrides (mainly for testing): MARIADB (client),
MARIADBD (the --prepare bootstrap server) and TAR (the tar
implementation, e.g. TAR=bsdtar) can each be overridden. To run the
bootstrap under rr, put it in MARIADBD and let rr's own _RR_TRACE_DIR
choose the trace location, e.g.
_RR_TRACE_DIR=/dev/shm/rr MARIADBD='rr record mariadbd'

--prepare
=========
Starts "mariadbd --bootstrap" on the backup directory using
backup-prepare.cnf as its defaults file, replays the archived redo
log between the start and target LSN read from backup.cnf,
then builds a fresh ib_logfile0 so a normal server can start
on the directory. mariadbd is taken from the path recorded in
backup-prepare.cnf if that binary exists, otherwise from PATH.
User --defaults-file/-extra-file and encryption options are
layered onto the bootstrap.

--copy-back / --move-back
=========================
Copy or move a prepared backup into the datadir. The datadir
is created if missing, a non-empty datadir is refused unless
--force-non-empty-directories is given, and a chown
reminder is printed.

If --aria-log-dir-path is given, the Aria logs (aria_log_control,
aria_log.*) are relocated into that directory.

Packaging
=========
The wrapper is not installed by default and never replaces the
real mariadb-backup / mbstream binaries.
1. cmake -DWITH_MARIABACKUP_WRAPPER=ON (default OFF) controls it.
2. When ON, the scripts install as /usr/bin/mariadb-backup-server
and /usr/bin/mbstream-server, tagged COMPONENT Backup so they
ship in the mariadb-backup package.
3. RPM: nothing extra to do. the component handles it.
4. DEB: not wired. debian/rules uses --fail-missing and does not
enable the option, so the -server binaries are not listed.
To ship via DEB, make a paired change: add
-DWITH_MARIABACKUP_WRAPPER=ON in debian/rules and list both
usr/bin/mariadb-backup-server and
usr/bin/mbstream-server in debian/mariadb-backup.install together.
5. The real mariadb-backup/mbstream binaries and the
mariabackup symlink are left untouched; opt in via an alias or a
symlink early in PATH.

Limitations (not supported yet)
===============================
1) Incremental backup & prepare (--incremental-basedir,
--incremental-dir, --apply-log-only)
2) --rollback-xa
3) Partial backup (--databases, --tables, --tables-file)
4) Output compression and encryption (--compress, --encrypt)
5) --export is accepted but only warns and runs a plain recovery
6) --extra-lsndir is ignored
7) Windows: POSIX sh only, not installed on Windows

Behaviour differences from native mariadb-backup
================================================
- The wrapper needs the mariadb client on PATH for
--backup, and mariadbd on PATH (or recorded in backup-prepare.cnf)
for --prepare
- BACKUP SERVER refuses an already-existing target directory
- BACKUP SERVER does copy the data file as raw pages without
checksum validation, so a corrupted table is not detected
at backup time
- --prepare only works on a wrapper-made backup. It
needs backup-prepare.cnf)
- --stream is tar, not xbstream, and local-only

Tests
=====
include/have_mariabackup_wrapper.inc redirects $XTRABACKUP to
mariadb-backup-server.sh and $XBSTREAM to mbstream-server.sh, skipping
when a wrapper or the mariadb client is unavailable.
include/have_mariabackup_combination.inc runs a test under both the
[CLIENT] mariadb-backup binary and the [SERVER] wrapper.

fe1f0a2e68e5...

fe1f0a2e68e5ec0d6801fcd993e86dfe716bf551

Marko Mäkelä

marko.makela@mariadb.com

fixup! 6c59b142f4acb4a5a7477358a25c2a56ec915386

f41aca23bf03...

f41aca23bf030f285a21edc963119db32681ac72

Daniel Black

daniel@mariadb.org

MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb` (2)

my_numchars_mb and my_charpos_mb both used their cs argument
so remove the notused attribute on them.

ed5f6c3b64dc...

ed5f6c3b64dca445d5c0e629b9999076d6d1dbdb

Alexey (Holyfoot) Botchkov

holyfoot@mariadb.com

MDEV-40013 Server crashes in XMLSchema_user_type::validate_prepare/XMLSchema_attribute::validate_prepare.

complexType doesn't have to have any compositor at all. So let's set
'empty_compositor' in this case.

e3dd55c0d195...

e3dd55c0d1953a1be1a14680c400630f325d7e9a

Raghunandan Bhat

raghunandan.bhat96@gmail.com

MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb`

Problem:
When `my_charpos_mb()` is called with pos = end = NULL and the string
has fewer than `length` characters, the `end + 2 - start` return
expression evaluates `end+2`, forming the pointer NULL+2. Offsetting
a null pointer is undefined behavior.

Fix:
Compute the integer difference before adding the offset. The result is
identical but no invalid pointer is ever formed.

de383961cda8...

de383961cda87c10f45a142b88aa534e81c575b6

Georg Richter

georg@mariadb.com

Fixed infer NULL dereference reports in unittests

cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed - stdio

cdad3e58ea43...

cdad3e58ea43321c88647f530540bd8244bf50ef

Marko Mäkelä

marko.makela@mariadb.com

squash! 81b3ae71537ca4c67ea4d0f740778f1596fd29a8

InnoDB_backup::queue: Collection of tablespace IDs and payload sizes
at the start of the backup

b9e7173125c0...

b9e7173125c016fbe0948a991a5a6fb3843646ee

Yuchen Pei

ycp@mariadb.com

MDEV-40103 Initialise thd->net in spider_create_sys_thd

net is used by THD::print_aborted_warning when threads are killed so
this needed to be initalised.

This fixes an msan "uninitialized value" bug

b975697da11e...

b975697da11ed62cbce7e528e3661e5e4ac1d81e

Yuchen Pei

ycp@mariadb.com

ac4c5139affa...

ac4c5139affa55cb82cf2f10e96f53727278d1e5

Thirunarayanan Balathandayuthapani

thiru@mariadb.com

a98a0687e0af...

a98a0687e0af52fe19d24dcd1aff305db8990190

Georg Richter

georg@mariadb.com

Fixed another infer NULL dereference report

cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed - stdio

a56133a25179...

a56133a251799e461dfa3d94e432a5bccc576d90

Yuchen Pei

ycp@mariadb.com

MDEV-39558 Check resulting VECTOR length in type aggregation inference

VECTOR, as a subtype of VARCHAR, has max length 65532, i.e. maximum
dimension of 16383. BLOB/MEDIUMBLOB/LONGBLOB (and corresponding TEXT
types) each has length exceeding 65532. Therefore, when aggregating
VECTOR with one of these BLOB/TEXT types, the aggregated type has
length exceeding the max length of VECTOR, which should result in an
error.

To that end, in this patch we add checks on the resulting vector
length during type aggregation inference

a485de5fdb0e...

a485de5fdb0e43a9f942aba0d36f1d23a8d9ff3e

Yuchen Pei

ycp@mariadb.com

MDEV-40103 Initialise thd->net in spider_create_sys_thd

This fixes an msan uninitialized bug

972e1cbaafd2...

972e1cbaafd2c4a151135090713959e0e4a5706c

ParadoxV5

paradox.ver5@gmail.com

MDEV-39788: Fix `master.info` not upgrading `master_use_gtid`

The line-count line in `master.info` and `relay-log.info`
has been inconsistent (off by one) since its introduction.
MDEV-37530 corrected this by changing `master.info`
to use `relay-log.info`’s line-count definition by
chance, but at the cost of an upgrade incompatibility.
It now reads one more line beyond the pre-upgraded file’s line-based
section, which for MariaDB 10.0–12.2 means an upgrade doesn’t carry
over the first option stored in `key=value`, `master_use_gtid`.

Since older versions can read the new format in a downgrade,
rather than encouraging the old inconsistency,
this commit focuses on the upgrade problem by adding a shim entry to
`master.info`’s list to emulate prior versions’ reading behaviour.
Although this implementation can only restore compatibility with
versions 10.0+, versions before MariaDB 10 have long been EOL.

While here, this commit also fixes code and
comments that contradict the actual effect.

94614c58c779...

94614c58c77929b9f1cf250d92eba7446cf929ba

drrtuy

drrtuy@gmail.com

fix(perf): optimize regexp_rewrite perf for CB.

93a6b989d3ca...

93a6b989d3ca3ef9ff64181f598edc55db5157ab

Alessandro Vetere

iminelink@gmail.com

MDEV-40209 Escalate lock-release via a saturating stall counter

lock_release() and lock_release_on_prepare() release a committing or preparing
transaction's explicit locks under the shared lock_sys.rd_lock(), taking each
per-cell hash latch and per-table lock mutex with a trylock because trx->mutex
is held in the reverse of the normal latch order. A single failed trylock
marked the whole pass unsuccessful, and after a fixed cap of 5 such passes the
code escalated to the exclusive lock_sys.wr_lock() for the whole transaction.
Under concurrency the trylocks fail transiently, so the cap escalated
transactions that were still steadily releasing locks, not just stuck ones; the
exclusive latch then blocks every concurrent lock_sys.rd_lock() acquirer in
lock_rec_lock() and lock_table(), producing a convoy. The chance of hitting the
cap rises with both the contention level and the number of latches a
transaction must trylock per pass.

Replace the fixed cap with a saturating stall counter (LOCK_RELEASE_MAX_STALLS,
incremented on a no-progress pass, decremented on progress, floored at zero)
that escalates a genuinely stuck transaction after 5 net stalls, as the fixed
cap did, while leaving a transaction that keeps making progress to finish under
the shared latch. A hard LOCK_RELEASE_MAX_PASSES ceiling bounds the loop
independently, for the case where concurrent activity keeps adding locks (e.g.
implicit-to-explicit conversion during XA PREPARE) so that progress never
converges. The _try functions report progress through an out-parameter computed
under trx->mutex, so trx->lock.trx_locks is never read unlatched.

8f2880e41491...

8f2880e414916530e67a1d0f8b3722072160829f

Yuchen Pei

ycp@mariadb.com

85bf32fe1c1d...

85bf32fe1c1d91c61974753fdb3fe220242e6247

Alessandro Vetere

iminelink@gmail.com

MDEV-40210 Redundant CAS in async_flush_lsn::try_clear_if_at_most()

MDEV-39600 added try_clear_if_at_most() to clear buf_flush_async_lsn with
an atomic CAS that preserves a concurrent bump(). If the snapshot is
already 0, compare_exchange_strong(0, 0) is a no-op, so return early on a
zero snapshot and avoid the atomic read-modify-write. The page cleaner
calls this on every pass, so in the common steady state (no async flush
queued) it drops needless exclusive access to the m_lsn cache line. A zero
value is already the cleared state and a concurrent bump() is preserved
either way, so the result is identical.

838a896b43db...

838a896b43db5fd6dcb9593a29d8718cc42bc6f3

drrtuy

drrtuy@gmail.com

chore(docs): architecture and shared resources docs.

7513fb64afdd...

7513fb64afddb8285f29163a8788e8b95a6b36ad

Daniel Black

daniel@mariadb.org

MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb` (2)

my_numchars_mb and my_charpos_mb both used their cs argument
so remove the notused attribute on them.

6a25d5bdc786...

6a25d5bdc786a58919957ac8703d0168685cad6e

drrtuy

drrtuy@gmail.com

fix: use jemalloc as a default allocator for DuckDB.

5e70c0637eef...

5e70c0637eefc470c7cd7fd896467dcd7bc4fe59

Sergei Golubchik

serg@mariadb.org

MDEV-39776 SHOW CREATE TABLE corrupts `-quoted table options

Two problems:

1. option value specified as `foo bar` was not quoted,
printed as foo bar, breaking CREATE TABLE.
2. Option value that included */ was printed as is, in unknown
options when the whole option was commented out, this broke the
comment: /* opt='val*/ue' */

Fix: simplify value printing, ignore the original quoting,
always print the value quoted as a string. Break comment markers
with an escape (as *\/) if inside a comment.

Note, cannot use `-quoting, because \-escapes don't work inside,
backticks, so */ cannot be broken.

584f7ab0f573...

584f7ab0f573a65c562592b3bb42dfff7854ce3e

Yuchen Pei

ycp@mariadb.com

547a8019abd8...

547a8019abd8aadb321ffd5f904de9119dac7b83

Vladislav Vaintroub

vvaintroub@gmail.com

MDEV-40026 acl_insert_db crashes on overlong dbname in "denies" array.

Fix to return error when json_unescape() fails.
Also return error, if length from json_unescape happens to be larger
that size of buffer - 1, to make sure that null-termination of string
does not override memory.

Some cleanup - only complain once about invalid "denies" entry during
FLUSH privileges. Avoid full parsing during when processing
mysql.global_priv in grant_reload() in the first pass.

4928129c47a9...

4928129c47a9b93281a757c39ee72878e899d789

Vladislav Vaintroub

vvaintroub@gmail.com

MDEV-40116 Column-level DENY SELECT hides accessible columns

Allow SHOW if grants on table level indicate "maybe allowed", i.e
there are some grants that are not all denied on this level.

Column level denies are applied in SHOW COLUMS.
They are not applied in SHOW CREATE TABLE, which will succeed if
there are appropriate table level privs, without checking columns.

44bfe1a06ed9...

44bfe1a06ed943db9574ae536367cc4b17a85ced

Yuchen Pei

ycp@mariadb.com

MDEV-40103 Initialise thd->net in spider_create_sys_thd

This fixes an msan uninitialized bug

4243cc27afa7...

4243cc27afa7ffe5297b8d55cab55b988dad030d

drrtuy

drrtuy@gmail.com

chore(test): TPC-H gen outputs in parquet format and queries run by MariaDB native SQL.

3cb7949cbc88...

3cb7949cbc88771fad63b8553de570f96c5f202f

Yuchen Pei

ycp@mariadb.com

MDEV-40088 [to-squash] Disallow subqueries in INTERVAL clause in range interval auto partitioning

This is consistent with system time (versioning) partitioning. Also
consistent is that both allow expressions otherwise.

340a8c612bdd...

340a8c612bddcc37f925f1c82a9e499d425701c6

Georg Richter

georg@mariadb.com

Merge branch '3.3' into 3.4

cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed - stdio

291c427f99f9...

291c427f99f96361722a71f68ccedf81fd9e7ed5

Vladislav Vaintroub

vvaintroub@gmail.com

MDEV-40115 DENY EXECUTE hides unrelated routines in SHOW commands or I_S

Fix check_routine_level_acl() to pass parent access, for accurate
calculation.

Previously, if (db_level_access & privs) returned 0, it was treated as
no-access on DB level in check_some_routine_access().

This is not the complete truth, with DENY, it may mean there is access,
but some routines have DENY. So pass the db level grants/denies to the
function for more accurate checking.

257d17a4cce3...

257d17a4cce33528b2287e3a87e7a8b6077a762c

Sergei Golubchik

serg@mariadb.org

MDEV-39818 backtick in FK names breaks SHOW CREATE TABLE

InnoDB used to have two versions of innobase_quote_identifier().
One that handled embedded backticks and one that didn't.

Let's use just one implementation that handles backticks.
Keep the other one as a helper that writes to FILE but does not
implement the quoting logic.

2377981cccf3...

2377981cccf3d69cc54da7e90dfb65a63582235d

Yuchen Pei

ycp@mariadb.com

1dc6a4800cba...

1dc6a4800cba9b332ff67c36acb2b78367aa390f

ParadoxV5

paradox.ver5@gmail.com

Merge branch 'MDEV-39788.test' into MDEV-39788

1b346f46af8a...

1b346f46af8aa0a583c794c2faa15e52bc31bfc8

Yuchen Pei

ycp@mariadb.com

MDEV-40103 Initialise thd->net in spider_create_sys_thd

This fixes an msan uninitialized bug

18352a96ab2f...

18352a96ab2fdcb512f771a80435af65d44d3b33

Alexey (Holyfoot) Botchkov

holyfoot@mariadb.com

MDEV-40019 Xmlisvalid shows wrong result if calling it multiple times from query with UNION ALL.

Call the validation_prepare() for the tail of the list.

0b09764d32e0...

0b09764d32e0b30ec1c40e30f6c7e318793a06c4

Vladislav Vaintroub

vvaintroub@gmail.com

MDEV-40028 Assertion `rights.allow_bits() == merged->cols' failed

Fixed mismatched memroot in new operator, in apply_deny_column.

MDEV-40014 Crash in SHOW GRANTS FOR user after REVOKE

Fixed premature my_hash_reset in update_role_columns, should not be
done if only DENYs are present

Also fixed unintended GRANT on column privileges when DENY should be
applied.

06ae6b7056a3...

06ae6b7056a3917a9bd8a79995ae7e5c3ee3ac3a

Raghunandan Bhat

raghunandan.bhat96@gmail.com

05d999dbcf22...

05d999dbcf2245f8192a41cac175ca514d51f00e

Georg Richter

georg@mariadb.com

Fixed another issue found by rpl_fuzzer

cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed - stdio

03c114f1ca1d...

03c114f1ca1d9eaba45104dbd233c7b9dc3d0a77

Vladislav Vaintroub

vvaintroub@gmail.com

MDEV-40131 DENY granted on roles exclusively is missing after FLUSH PRIVILEGES.

Fixed: user_table_has_denies was not set for role entries, but for normal
users only.