Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Thirunarayanan Balathandayuthapani
MDEV-39061 mariadb-backup compatible wrapper for BACKUP SERVER

This adds a shell script that lets users keep using their existing
mariadb-backup commands while the real work is done by the new
server-side BACKUP SERVER command. The goal is "drop-in": users should
not have to change their backup scripts.

extra/mariabackup/scripts/mariadb-backup-server.sh (plain POSIX sh)
understands the usual mariadb-backup modes and translates each one.
A companion helper, extra/mariabackup/scripts/mbstream-server.sh,
lets streamed backups be unpacked by pipelines that expect the
mbstream CLI. Both are documented in
extra/mariabackup/scripts/README.md.

--backup
========
Connects with the mariadb client and runs "BACKUP SERVER TO '<dir>'".
Connection options (--user, --host, --port, --socket, --defaults-file,
ssl, ...) are passed through to the client; --parallel=N becomes the
"<N> CONCURRENT" clause.
After the backup it writes backup-prepare.cnf into the backup
directory, recording what --prepare needs later: where
mariadbd lives, the InnoDB parameters (page size, data file path,
undo tablespaces, checksum algorithm, log file size), and if
the server is encrypted then how to reload the encryption key
plugin (the file_key_management variables),
so an encrypted backup can be prepared without extra input.

--backup --stream
=================
Runs "BACKUP SERVER WITH [N CONCURRENT] '<command>'": the server feeds
each stream's tar to <command>, the wrapper collects the parts, writes
them to stdout, then appends backup-prepare.cnf as a final tar. The
per-stream tars carry no end-of-archive marker; only the trailing
backup-prepare.cnf adds the single end marker, so the whole stream
extracts with a plain "tar -x".
Two properties follow from how BACKUP SERVER streams,
both differing from mariadb-backup:
- local: the stream command runs inside the server, so the wrapper
  must share its filesystem;
- tar only: any --stream=<format> (including xbstream) yields tar.
--target-dir is optional in stream mode (scratch for the per-stream
parts; a mktemp dir is used otherwise).

mbstream-server.sh maps the mbstream CLI onto a plain "tar -x"/"tar -c", so
existing "mbstream -x"/"-c" pipelines keep working on the wrapper's
stream. mbstream-only flags (-p/--parallel, ...) are accepted and
ignored; any other unknown option is rejected.

Environment overrides (mainly for testing): MARIADB (client),
MARIADBD (the --prepare bootstrap server) and TAR (the tar
implementation, e.g. TAR=bsdtar) can each be overridden. To run the
bootstrap under rr, put it in MARIADBD and let rr's own _RR_TRACE_DIR
choose the trace location, e.g.
  _RR_TRACE_DIR=/dev/shm/rr MARIADBD='rr record mariadbd'

--prepare
=========
Starts "mariadbd --bootstrap" on the backup directory using
backup-prepare.cnf as its defaults file, replays the archived redo
log between the start and target LSN read from backup.cnf,
then builds a fresh ib_logfile0 so a normal server can start
on the directory. mariadbd is taken from the path recorded in
backup-prepare.cnf if that binary exists, otherwise from PATH.
User --defaults-file/-extra-file and encryption options are
layered onto the bootstrap.

--copy-back / --move-back
=========================
Copy or move a prepared backup into the datadir. The datadir
is created if missing, a non-empty datadir is refused unless
--force-non-empty-directories is given, and a chown
reminder is printed.

If --aria-log-dir-path is given, the Aria logs (aria_log_control,
aria_log.*) are relocated into that directory.

Packaging
=========
The wrapper is not installed by default and never replaces the
real mariadb-backup / mbstream binaries.
1. cmake -DWITH_MARIABACKUP_WRAPPER=ON (default OFF) controls it.
2. When ON, the scripts install as /usr/bin/mariadb-backup-server
and /usr/bin/mbstream-server, tagged COMPONENT Backup so they
ship in the mariadb-backup package.
3. RPM: nothing extra to do. the component handles it.
4. DEB: not wired. debian/rules uses --fail-missing and does not
enable the option, so the -server binaries are not listed.
To ship via DEB, make a paired change: add
-DWITH_MARIABACKUP_WRAPPER=ON in debian/rules and list both
usr/bin/mariadb-backup-server and
usr/bin/mbstream-server in debian/mariadb-backup.install together.
5. The real mariadb-backup/mbstream binaries and the
mariabackup symlink are left untouched; opt in via an alias or a
symlink early in PATH.

Limitations (not supported yet)
===============================
1) Incremental backup & prepare (--incremental-basedir,
  --incremental-dir, --apply-log-only)
2) --rollback-xa
3) Partial backup (--databases, --tables, --tables-file)
4) Output compression and encryption (--compress, --encrypt)
5) --export is accepted but only warns and runs a plain recovery
6) --extra-lsndir is ignored
7) Windows: POSIX sh only, not installed on Windows

Behaviour differences from native mariadb-backup
================================================
- The wrapper needs the mariadb client on PATH for
--backup, and mariadbd on PATH (or recorded in backup-prepare.cnf)
for --prepare
- BACKUP SERVER refuses an already-existing target directory
- BACKUP SERVER does copy the data file as raw pages without
checksum validation, so a corrupted table is not detected
at backup time
- --prepare only works on a wrapper-made backup. It
needs backup-prepare.cnf)
- --stream is tar, not xbstream, and local-only

Tests
=====
include/have_mariabackup_wrapper.inc redirects $XTRABACKUP to
mariadb-backup-server.sh and $XBSTREAM to mbstream-server.sh, skipping
when a wrapper or the mariadb client is unavailable.
include/have_mariabackup_combination.inc runs a test under both the
[CLIENT] mariadb-backup binary and the [SERVER] wrapper.
Raghunandan Bhat
MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb`

Problem:
  When `my_charpos_mb()` is called with pos = end = NULL and the string
  has fewer than `length` characters, the `end + 2 - start` return
  expression evaluates `end+2`, forming the pointer NULL+2. Offsetting
  a null pointer is undefined behavior.

Fix:
  Compute the integer difference before adding the offset. The result is
  identical but no invalid pointer is ever formed.
Daniel Black
MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb` (2)

my_numchars_mb and my_charpos_mb both used their cs argument
so remove the notused attribute on them.
Alexey (Holyfoot) Botchkov
MDEV-40013 Server crashes in XMLSchema_user_type::validate_prepare/XMLSchema_attribute::validate_prepare.

complexType doesn't have to have any compositor at all. So let's set
'empty_compositor' in this case.
Raghunandan Bhat
MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb`

Problem:
  When `my_charpos_mb()` is called with pos = end = NULL and the string
  has fewer than `length` characters, the `end + 2 - start` return
  expression evaluates `end+2`, forming the pointer NULL+2. Offsetting
  a null pointer is undefined behavior.

Fix:
  Compute the integer difference before adding the offset. The result is
  identical but no invalid pointer is ever formed.
Georg Richter
Fixed infer NULL dereference reports in unittests
  • cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed -  stdio
Vladislav Vaintroub
MDEV-14443 DENY statement

Implements DENY/REVOKE DENY and associated tasks.
Rucha Deodhar
MDEV-31477: Inconsistent handling while fetching values in json

Analysis:
When we get value in json_value(), if any error occurs while scanning the
json, it is not reported. Only true is returned.
Fix:
Report error along with returning true (error)
forkfun
MDEV-39380 Assertion `arg2_int >= 0' failed in Item_func_additive_op::result_precision

SUM() of an argument with no fixed decimal scale (NOT_FIXED_DEC, e.g. a
string/bit-literal via COALESCE) used the raw args[0]->decimals as the
result scale, clamping it to DECIMAL_MAX_SCALE while deriving precision
separately. This yielded a DECIMAL with scale > precision, which then
tripped the assert in Item_func_additive_op::result_precision()

Use args[0]->decimal_scale() instead, which bounds NOT_FIXED_DEC to
MY_MIN(max_length, DECIMAL_MAX_SCALE) so scale <= precision.
Thirunarayanan Balathandayuthapani
MDEV-39061 mariadb-backup compatible wrapper for BACKUP SERVER

This adds a shell script that lets users keep using their existing
mariadb-backup commands while the real work is done by the new
server-side BACKUP SERVER command. The goal is "drop-in": users should
not have to change their backup scripts.

extra/mariabackup/scripts/mariadb-backup-server.sh (plain POSIX sh)
understands the usual mariadb-backup modes and translates each one.
A companion helper, extra/mariabackup/scripts/mbstream-server.sh,
lets streamed backups be unpacked by pipelines that expect the
mbstream CLI. Both are documented in
extra/mariabackup/scripts/README.md.

--backup
========
Connects with the mariadb client and runs "BACKUP SERVER TO '<dir>'".
Connection options (--user, --host, --port, --socket, --defaults-file,
ssl, ...) are passed through to the client; --parallel=N becomes the
"<N> CONCURRENT" clause.
After the backup it writes backup-prepare.cnf into the backup
directory, recording what --prepare needs later: where
mariadbd lives, the InnoDB parameters (page size, data file path,
undo tablespaces, checksum algorithm, log file size), and if
the server is encrypted then how to reload the encryption key
plugin (the file_key_management variables),
so an encrypted backup can be prepared without extra input.

--backup --stream
=================
Runs "BACKUP SERVER WITH [N CONCURRENT] '<command>'": the server feeds
each stream's tar to <command>, the wrapper collects the parts, writes
them to stdout, then appends backup-prepare.cnf as a final tar. The
per-stream tars carry no end-of-archive marker; only the trailing
backup-prepare.cnf adds the single end marker, so the whole stream
extracts with a plain "tar -x".
Two properties follow from how BACKUP SERVER streams,
both differing from mariadb-backup:
- local: the stream command runs inside the server, so the wrapper
  must share its filesystem;
- tar only: any --stream=<format> (including xbstream) yields tar.
--target-dir is optional in stream mode (scratch for the per-stream
parts; a mktemp dir is used otherwise).

mbstream-server.sh maps the mbstream CLI onto a plain "tar -x"/"tar -c", so
existing "mbstream -x"/"-c" pipelines keep working on the wrapper's
stream. mbstream-only flags (-p/--parallel, ...) are accepted and
ignored; any other unknown option is rejected.

Environment overrides (mainly for testing): MARIADB (client),
MARIADBD (the --prepare bootstrap server) and TAR (the tar
implementation, e.g. TAR=bsdtar) can each be overridden. To run the
bootstrap under rr, put it in MARIADBD and let rr's own _RR_TRACE_DIR
choose the trace location, e.g.
  _RR_TRACE_DIR=/dev/shm/rr MARIADBD='rr record mariadbd'

--prepare
=========
Starts "mariadbd --bootstrap" on the backup directory using
backup-prepare.cnf as its defaults file, replays the archived redo
log between the start and target LSN read from backup.cnf,
then builds a fresh ib_logfile0 so a normal server can start
on the directory. mariadbd is taken from the path recorded in
backup-prepare.cnf if that binary exists, otherwise from PATH.
User --defaults-file/-extra-file and encryption options are
layered onto the bootstrap.

--copy-back / --move-back
=========================
Copy or move a prepared backup into the datadir. The datadir
is created if missing, a non-empty datadir is refused unless
--force-non-empty-directories is given, and a chown
reminder is printed.

If --aria-log-dir-path is given, the Aria logs (aria_log_control,
aria_log.*) are relocated into that directory.

Packaging
=========
The wrapper is not installed by default and never replaces the
real mariadb-backup / mbstream binaries.
1. cmake -DWITH_MARIABACKUP_WRAPPER=ON (default OFF) controls it.
2. When ON, the scripts install as /usr/bin/mariadb-backup-server
and /usr/bin/mbstream-server, tagged COMPONENT Backup so they
ship in the mariadb-backup package.
3. RPM: nothing extra to do. the component handles it.
4. DEB: not wired. debian/rules uses --fail-missing and does not
enable the option, so the -server binaries are not listed.
To ship via DEB, make a paired change: add
-DWITH_MARIABACKUP_WRAPPER=ON in debian/rules and list both
usr/bin/mariadb-backup-server and
usr/bin/mbstream-server in debian/mariadb-backup.install together.
5. The real mariadb-backup/mbstream binaries and the
mariabackup symlink are left untouched; opt in via an alias or a
symlink early in PATH.

Limitations (not supported yet)
===============================
1) Incremental backup & prepare (--incremental-basedir,
  --incremental-dir, --apply-log-only)
2) --rollback-xa
3) Partial backup (--databases, --tables, --tables-file)
4) Output compression and encryption (--compress, --encrypt)
5) --export is accepted but only warns and runs a plain recovery
6) --extra-lsndir is ignored
7) Windows: POSIX sh only, not installed on Windows

Behaviour differences from native mariadb-backup
================================================
- The wrapper needs the mariadb client on PATH for
--backup, and mariadbd on PATH (or recorded in backup-prepare.cnf)
for --prepare
- BACKUP SERVER refuses an already-existing target directory
- BACKUP SERVER does copy the data file as raw pages without
checksum validation, so a corrupted table is not detected
at backup time
- --prepare only works on a wrapper-made backup. It
needs backup-prepare.cnf)
- --stream is tar, not xbstream, and local-only

Tests
=====
include/have_mariabackup_wrapper.inc redirects $XTRABACKUP to
mariadb-backup-server.sh and $XBSTREAM to mbstream-server.sh, skipping
when a wrapper or the mariadb client is unavailable.
include/have_mariabackup_combination.inc runs a test under both the
[CLIENT] mariadb-backup binary and the [SERVER] wrapper.
Georg Richter
Fixed another infer NULL dereference report
  • cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed -  stdio
Vladislav Vaintroub
MDEV-37187 hashicorp plugin: avoid expensive clock() in hot path

The plugin used clock() only to timestamp cache entries and measure
elapsed time against millisecond timeouts. As shown in MDEV-12345,
clock() is prohibitively expensive, and we do not need per-thread or
per-process CPU time here, only time differences.

Replace clock()/clock_t with the monotonic std::chrono::steady_clock.
Timeouts are kept as std::chrono::milliseconds
drrtuy
fix(perf): optimize regexp_rewrite perf for CB.
Alessandro Vetere
MDEV-40209 Escalate lock-release via a saturating stall counter

lock_release() and lock_release_on_prepare() release a committing or preparing
transaction's explicit locks under the shared lock_sys.rd_lock(), taking each
per-cell hash latch and per-table lock mutex with a trylock because trx->mutex
is held in the reverse of the normal latch order. A single failed trylock
marked the whole pass unsuccessful, and after a fixed cap of 5 such passes the
code escalated to the exclusive lock_sys.wr_lock() for the whole transaction.
Under concurrency the trylocks fail transiently, so the cap escalated
transactions that were still steadily releasing locks, not just stuck ones; the
exclusive latch then blocks every concurrent lock_sys.rd_lock() acquirer in
lock_rec_lock() and lock_table(), producing a convoy. The chance of hitting the
cap rises with both the contention level and the number of latches a
transaction must trylock per pass.

Replace the fixed cap with a saturating stall counter (LOCK_RELEASE_MAX_STALLS,
incremented on a no-progress pass, decremented on progress, floored at zero)
that escalates a genuinely stuck transaction after 5 net stalls, as the fixed
cap did, while leaving a transaction that keeps making progress to finish under
the shared latch. A hard LOCK_RELEASE_MAX_PASSES ceiling bounds the loop
independently, for the case where concurrent activity keeps adding locks (e.g.
implicit-to-explicit conversion during XA PREPARE) so that progress never
converges. The _try functions report progress through an out-parameter computed
under trx->mutex, so trx->lock.trx_locks is never read unlatched.
Alessandro Vetere
MDEV-40210 Redundant CAS in async_flush_lsn::try_clear_if_at_most()

MDEV-39600 added try_clear_if_at_most() to clear buf_flush_async_lsn with
an atomic CAS that preserves a concurrent bump(). If the snapshot is
already 0, compare_exchange_strong(0, 0) is a no-op, so return early on a
zero snapshot and avoid the atomic read-modify-write. The page cleaner
calls this on every pass, so in the common steady state (no async flush
queued) it drops needless exclusive access to the m_lsn cache line. A zero
value is already the cleared state and a concurrent bump() is preserved
either way, so the result is identical.
drrtuy
chore(docs): architecture and shared resources docs.
Daniel Black
MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb` (2)

my_numchars_mb and my_charpos_mb both used their cs argument
so remove the notused attribute on them.
Marko Mäkelä
squash! 81b3ae71537ca4c67ea4d0f740778f1596fd29a8

Observe aria_log_dir_path

Patch based on code by Thirunarayanan Balathandayuthapani
Vladislav Vaintroub
MDEV-39951 DENY feature does not work for EVENT privilege

Fix calculation of need_table_or_column_check in check_access(),
so it does not consider already denied bits on DB level.

Add a test case for events (provided by Ramesh Sivaraman)
drrtuy
fix: use jemalloc as a default allocator for DuckDB.
Sergei Golubchik
MDEV-39776 SHOW CREATE TABLE corrupts `-quoted table options

Two problems:

1. option value specified as `foo bar` was not quoted,
  printed as foo bar, breaking CREATE TABLE.
2. Option value that included */ was printed as is, in unknown
  options when the whole option was commented out, this broke the
  comment: /* opt='val*/ue' */

Fix: simplify value printing, ignore the original quoting,
always print the value quoted as a string. Break comment markers
with an escape (as *\/) if inside a comment.

Note, cannot use `-quoting, because \-escapes don't work inside,
backticks, so */ cannot be broken.
Vladislav Vaintroub
MDEV-40026 acl_insert_db crashes on overlong dbname in "denies" array.

Fix to return error when json_unescape() fails.
Also return error, if length from json_unescape happens to be larger
that size of buffer - 1, to make sure that null-termination of string
does not override memory.

Some cleanup - only complain once about invalid "denies" entry during
FLUSH privileges. Avoid full parsing during when processing
mysql.global_priv in grant_reload() in the first pass.
Vladislav Vaintroub
MDEV-40002 REVOKE DENY on table debug asserts with existing GRANT, without existing DENY

Fix handling of update_denies_in_user_table() inside replace_table_table()

replace_table_table is "special", and should return  -1 if  grant was
revoked, 1 if error, 0 if success. Whenever it is used, caller explicitly
checks for return code greater 0

It did however return -1 on all errors with denies. The debug assertion
that catches cases where my_error() is followed by my_ok() fired then.

This is now fixed. Added test cases for "REVOKE DENY on non-existing DENY
and existing GRANT" scenarios.
Alessandro Vetere
MDEV-40129 Retry transient trylock failures in lock-release fast paths

The trylock attempts on per-cell lock_sys_t::hash_latch (try_acquire())
and on per-table dict_table_t::lock_mutex_trylock() inside
lock_release_try(), lock_release_on_prepare_try() and
lock_rec_unlock_unmodified() now use a bounded spin loop
(up to LOCK_RELEASE_TRY_SPIN_BUDGET CAS attempts, with MY_RELAX_CPU()
between them) instead of a single CAS attempt.

These paths hold trx->mutex while attempting the trylock, which is the
reverse of the standard order used by lock_rec_convert_impl_to_expl().
Blocking acquisition is therefore unsafe, hence the trylock pattern.
However, a single failed CAS marks the entire pass of lock_release_try()
as unsuccessful, and after 5 such failed passes lock_release() falls
back to exclusive lock_sys.wr_lock() for the whole transaction. That
global wr_lock then blocks every concurrent lock_sys.rd_lock() acquirer
in lock_rec_lock() and lock_table(), producing a server-wide convoy
under heavy concurrency.

The bounded spin (no syscall, no blocking) gives a transient latch
holder time to release without weakening the deadlock-avoidance
guarantee that motivated the trylock pattern. The extra trx->mutex hold
time is bounded by LOCK_RELEASE_TRY_SPIN_BUDGET times the pause cost.

This is a first, still to be fine-tuned implementation. Only the
lock_release_try() path has been positively tested; the
lock_release_on_prepare_try() path is not yet covered.
Vladislav Vaintroub
MDEV-40116 Column-level DENY SELECT hides accessible columns

Allow SHOW if grants on table level indicate "maybe allowed", i.e
there are some grants that are not all denied on this level.

Column level denies are applied in SHOW COLUMS.
They are not applied in SHOW CREATE TABLE, which will succeed if
there are appropriate table level privs, without checking columns.
drrtuy
chore(test): TPC-H gen outputs in parquet format and queries run by MariaDB native SQL.
Alessandro Vetere
MDEV-40129 Retry transient trylock failures in lock-release fast paths

The trylock attempts on per-cell lock_sys_t::hash_latch (try_acquire())
and on per-table dict_table_t::lock_mutex_trylock() inside
lock_release_try(), lock_release_on_prepare_try() and
lock_rec_unlock_unmodified() now use a bounded spin loop
(up to LOCK_RELEASE_TRY_SPIN_BUDGET CAS attempts, with MY_RELAX_CPU()
between them) instead of a single CAS attempt.

These paths hold trx->mutex while attempting the trylock, which is the
reverse of the standard order used by lock_rec_convert_impl_to_expl().
Blocking acquisition is therefore unsafe, hence the trylock pattern.
However, a single failed CAS marks the entire pass of lock_release_try()
as unsuccessful, and after 5 such failed passes lock_release() falls
back to exclusive lock_sys.wr_lock() for the whole transaction. That
global wr_lock then blocks every concurrent lock_sys.rd_lock() acquirer
in lock_rec_lock() and lock_table(), producing a server-wide convoy
under heavy concurrency.

The bounded spin (no syscall, no blocking) gives a transient latch
holder time to release without weakening the deadlock-avoidance
guarantee that motivated the trylock pattern. The extra trx->mutex hold
time is bounded by LOCK_RELEASE_TRY_SPIN_BUDGET times the pause cost.

This is a first, still to be fine-tuned implementation. Only the
lock_release_try() path has been positively tested; the
lock_release_on_prepare_try() path is not yet covered.
Georg Richter
Merge branch '3.3' into 3.4
  • cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed -  stdio
Alessandro Vetere
MDEV-40128 Use per-cell latch in lock_move_reorganize_page()

lock_move_reorganize_page() was acquiring lock_sys.latch in exclusive
mode (via LockMutexGuard) for the entire body of phase 2 (lock chain
iteration, bitmap reset, and lock_rec_add_to_queue() calls). The
function however only touches record locks belonging to a single page,
which all live in a single lock_sys.rec_hash cell. Holding that cell
latch in exclusive mode via LockGuard is sufficient:

- The cell latch protects the cell's lock chain and the bitmaps of the
  lock_t objects in it (lock_rec_bitmap_reset and the new bit set by
  lock_rec_add_to_queue()).
- It also protects lock->type_mode, including the LOCK_WAIT bit. The
  canonical clear in lock_reset_lock_and_trx_wait() runs under the cell
  latch, and lock_grant() invokes it before taking trx->mutex, so the bit
  is cell-latch state rather than trx->mutex state. Phase 1 only clears
  the bit and leaves trx->lock.wait_lock intact; the copy in old_locks
  keeps LOCK_WAIT and phase 2 re-adds the lock with it, so the wait
  relationship (guarded by lock_sys.wait_mutex) is preserved across the
  move. Neither trx->mutex nor wait_mutex is required here.
- Each owning trx's mutex is acquired per-iteration to protect that trx's
  trx_locks list and lock_heap during lock_rec_add_to_queue().

The global exclusive latch was over-strong: it blocked every concurrent
lock_sys.rd_lock() acquirer in lock_rec_lock() and lock_table()
server-wide for the duration of the reorganize, contributing
disproportionately to the lock_sys.latch convoy under heavy concurrency.

The TMLockGuard fast-path empty check at the top of the function is
preserved; for cells with no locks the cost is still just a TSX-elided
read.
Vladislav Vaintroub
MDEV-40115 DENY EXECUTE hides unrelated routines in SHOW commands or I_S

Fix check_routine_level_acl() to pass parent access, for accurate
calculation.

Previously, if (db_level_access & privs) returned 0, it was treated as
no-access on DB level in check_some_routine_access().

This is not the complete truth, with DENY, it may mean there is access,
but some routines have DENY. So pass the db level grants/denies to the
function for more accurate checking.
Sergei Golubchik
MDEV-39818 backtick in FK names breaks SHOW CREATE TABLE

InnoDB used to have two versions of innobase_quote_identifier().
One that handled embedded backticks and one that didn't.

Let's use just one implementation that handles backticks.
Keep the other one as a helper that writes to FILE but does not
implement the quoting logic.
Alexander Barkov
MDEV-40155 Weak REF CURSOR without RETURN is not opened using a dynamic SQL statement

This statement:
  OPEN c FOR 'dynamic sql'
was only allowed for SYS_REFCURSOR.

Additionally allow it for REF CURSOR with no RETURN clause, e.g.:
  TYPE cur0_t IS REF CURSOR; -- No RETURN clause - OK for OPEN FOR

Note, REF CURSORs with RETURN clause are still not allowed for dynamic OPEN,
as expected.
Alexey (Holyfoot) Botchkov
MDEV-40019 Xmlisvalid shows wrong result if calling it multiple times from query with UNION ALL.

Call the validation_prepare() for the tail of the list.
Raghunandan Bhat
MDEV-39690: UBSAN: signed integer overflow in `my_strntoll_8bit`, `my_strntoll_mb2_or_mb4` during BLOB-to-integer conversion

Problem:
  When converting a string like '-9223372036854775808' to an integer,
  the parsed magnitude (2^63) equals `(ulonglong) LONGLONG_MIN` and is
  accepted as valid. The return expression then cast it to signed
  (LONGLONG_MIN) and negated it. Negating LONGLONG_MIN is signed integer
  overflow, i.e. undefined behaviour.

Fix:
  Negate in unsigned arithmetic, where wrap-around is well defined, and
  convert to signed once: (longlong) (0ULL - i). 0ULL - 2^63 wraps to
  `0x8000000000000000`, which as signed longlong is LONGLONG_MIN.
Vladislav Vaintroub
MDEV-40028 Assertion `rights.allow_bits() == merged->cols' failed

Fixed mismatched memroot in new operator, in apply_deny_column.

MDEV-40014 Crash in SHOW GRANTS FOR user after REVOKE

Fixed premature my_hash_reset in update_role_columns, should not be
done if only DENYs are present

Also fixed unintended GRANT on column privileges when DENY should be
applied.
Raghunandan Bhat
MDEV-40176: UBSAN: runtime error: applying non-zero offset in `my_charpos_mb`

Problem:
  When `my_charpos_mb()` is called with pos = end = NULL and the string
  has fewer than `length` characters, the `end + 2 - start` return
  expression evaluates `end+2`, forming the pointer NULL+2. Offsetting
  a null pointer is undefined behavior.

Fix:
  Compute the integer difference before adding the offset. The result is
  identical but no invalid pointer is ever formed.
Georg Richter
Fixed another issue found by rpl_fuzzer
  • cc-x-codbc-windows: 'dojob pwd if '3.4' == '3.4' ls win32/test SET TEST_DSN=master SET TEST_DRIVER=master SET TEST_PORT=3306 SET TEST_SCHEMA=odbcmaster if '3.4' == '3.4' cd win32/test if '3.4' == '3.4' ctest --output-on-failure' failed -  stdio
Vladislav Vaintroub
MDEV-40131 DENY granted on roles exclusively is missing after FLUSH PRIVILEGES.

Fixed: user_table_has_denies was not set for role entries, but for normal
users only.