Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Marko Mäkelä
Update backup to use the file copying interface

TODO: use it for directory scans

TODO: maria file creation
Marko Mäkelä
fixup! c66a08db5b1f0a28c9aa2ce1b84b8253d6afe33e
Daniel Bartholomew
bump the VERSION
Marko Mäkelä
mysys patch adapted from Monty

my_copy(): Make use of copy_file_range(2) and block-cloning.

my_copy_file(): Copy between my_native_file handles.

my_copy_file_range(): Copy a range between my_native_file handles.

my_clone_file_range(): Clone a range between my_native_file handles
(Linux only; FreeBSD does this on my_copy_file_range()).

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

TODO: Use these in backup
Alessandro Vetere
MDEV-41939 Rollback may leave a delete-marked secondary index record that purge does not remove

row_undo_mod_sec_is_unsafe() freezes purge_sys.view across the dereference
of externally stored columns, and under the freeze it tests whether purge
can see the transaction whose undo log record rebuilt the older version.
If the view advanced between the walk and the freeze, it reported the
secondary index entry as still needed, and the rollback delete-marked the
entry. No undo log record refers to that entry, so purge does not remove
it, unless a later write of the same key value reuses it.
CHECK TABLE ... EXTENDED reports it as missing its clustered index record
once purge_sys.end_view moves past the transaction that rolled back.

When purge can see that same transaction, trx_undo_prev_version_build()
ends the walk with no older version, and row_purge_is_unsafe() treats the
end of the versions in the same way. Report the entry as not needed in this
case too, so that the rollback removes it: no read view needs the rebuilt
version, and the walk found no match among the newer versions.

row_undo_mod_sec_is_unsafe_keep: a debug keyword that restores the previous
answer, as a negative control. With
--mysqld=--debug-dbug=+d,row_undo_mod_sec_is_unsafe_keep,
innodb.old_blob_purgeable fails with the orphan report.

innodb.old_blob_purgeable: force a purge batch after the rollback, so that
CHECK TABLE ... EXTENDED would report the orphan regardless of purge timing.
Use MEDIUMTEXT, because with innodb_page_size=64k the value
REPEAT('x', @@innodb_page_size) does not fit in TEXT.

innodb.old_blob_updel: wait until the rows that the test inserts first are
purged, before the test holds a purge batch. Otherwise purge_hold_cleanup
can hold the batch that purges the history of those rows, instead of the
batch that the test expects. This failed under Valgrind.

Assisted-by: Claude:claude-opus-5-5
Daniel Bartholomew
bump the VERSION
Marko Mäkelä
Move some platform differentiation to backup_target member functions
Marko Mäkelä
Move some platform differentiation to backup_target member functions
Marko Mäkelä
Update backup to use the file copying interface

TODO: use it for directory scans

TODO: maria file creation
sjaakola
MDEV-41597: Do privilege checks of triggers and views before TOI

CREATE TRIGGER, CREATE VIEW and ALTER VIEW did some of their privilege
checks only after TOI had replicated the statement. The applier runs
with all privileges, so a statement refused on the originating node
was still applied by the other nodes:
- the DEFINER clause (sp_process_definer(), which requires SET USER
  for a DEFINER other than the current user), for all three
  statements, creating the trigger or view with a DEFINER the user
  was not allowed to use
- the column privileges of a view (ER_COLUMNACCESS_DENIED_ERROR),
  creating a view that gives more privileges than the user has on
  the underlying tables

ALTER VIEW without a DEFINER clause also replicated the wrong
definition. fill_defined_view_parts(), which takes DEFINER, ALGORITHM
and SQL SECURITY from the existing view, ran after the replicated
statement had been built, so the other nodes got the current user as
DEFINER and the default SQL SECURITY.

Do the checks before TOI begins, as is already done for stored
routines and events:
- mysql_create_or_drop_trigger() calls sp_process_definer() for
  CREATE TRIGGER before WSREP_TO_ISOLATION_BEGIN, and
  Table_triggers_list::create_trigger() no longer calls it.
- mysql_create_view() calls fill_defined_view_parts() (ALTER VIEW)
  and sp_process_definer() early, and begins TOI only after all
  checks, just before mysql_register_view().

The order of checks changes for all builds: the DEFINER error is now
reported before some errors that were found later in the statement.

The test galera.galera_toi_definer_check checks that:
- CREATE TRIGGER, CREATE VIEW and ALTER VIEW refused for a foreign
  DEFINER on node_1 are not applied on node_2
- CREATE VIEW refused for missing column privileges on node_1 is not
  applied on node_2
- ALTER VIEW without a DEFINER clause keeps the same DEFINER and
  SQL SECURITY on both nodes

Assisted By: Claude Opus 5.5
Alexey Botchkov
How i see we can get rid of superfluous json manipulations.

We can make sure that the val_json() only returns valid JSONs.
Also we can add the 'nice' argument to the val_json() function
that would require the returning JSON be 'nice' as after the
json_nice(LOOSE) call.
Thirunarayanan Balathandayuthapani
MDEV-41494  Server aborts on crash recovery after an interrupted instant ALTER TABLE reorders/drops columns

Problem:
========
  After an instant ALTER TABLE that reorders or drops columns,
innobase_instant_try() deletes/updates SYS_COLUMNS, SYS_VIRTUAL
and SYS_TABLES.N_COLS, and only afterwards writes the hidden
metadata record that describes the old column layout in the
clustered index. These are separate mini-transactions within
the same DDL transaction.

If the server is killed after the SYS_COLUMNS/SYS_TABLES updates
were made durable but before the metadata record was written,
and recovery does not roll back the transaction then
table's dictionary definition and its on-disk metadata record go out
of sync.

On the next load, btr_cur_instant_init_low() correctly
detects this as "missing instant ALTER metadata" or "contains
unrecognizable instant ALTER metadata" and sets table->corrupted,
but it did not set table->file_unreadable. Since
dict_table_t::is_readable() only tests file_unreadable, the table
was left in an inconsistent corrupted-but-readable state.

Solution:
=========
btr_cur_instant_init_low(): Also set index->table->file_unreadable
in the "missing instant ALTER metadata" and "incompatible instant
ALTER metadata" error paths, consistent with every other corruption
path in this function.
Marko Mäkelä
mysys patch adapted from Monty

my_copy(): Make use of copy_file_range(2) and block-cloning.

my_copy_file(): Copy between my_native_file handles.

my_copy_file_range(): Copy a range between my_native_file handles.

my_clone_file_range(): Clone a range between my_native_file handles
(Linux only; FreeBSD does this on my_copy_file_range()).

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

TODO: Replace sql_backup.cc with these
Tarun Wadhwa
MDEV-37605: Extend mariadb-binlog to convert InnoDB-based binlogs to legacy format

MariaDB 12.3 added support for storing binary logs inside InnoDB
tablespace files (.ibb) instead of traditional flat sequential files.
There was previously no way to convert these back to the legacy
format, which is needed for downgrades, migrations, and third-party
tooling that only understands classic binlogs.

This adds a --convert-engine-binlog option to mariadb-binlog that
reads one or more InnoDB-based binlogs and writes out equivalent
legacy-format binlog files.

Usage:
  mariadb-binlog --convert-engine-binlog -r output_legacy_binlog \
      input1.ibb input2.ibb

This produces output_legacy_binlog.0000001, .0000002, etc. Note
there isn't a direct correlation between the number of input files
and output files. The output is split whenever the current file
hits --max-binlog-size (default 1GB, since legacy binlogs cannot
exceed 4GB), and also on every server restart recorded in the
InnoDB binlog, so a single input file can produce several output
files and a restart boundary can force a split even before the
size limit is reached.

Additionally, --max-binlog-size is also supported as a standalone
option to control this rotation threshold directly.

Only --convert-engine-binlog, -r, and --max-binlog-size are
supported alongside this mode; any other option will cause
mariadb-binlog to throw an error.

Caveats:
- Encryption: InnoDB-based binlogs don't currently support
  encryption, so the converter doesn't handle it either. Output
  legacy binlogs are never encrypted by this mode.
- Checksums: InnoDB-based binlogs rely on page-level checksums
  instead of the legacy per-event checksums, so the initial
  implementation generates legacy output with no event-level
  checksum.

Reviewed-by: Kristian Nielsen <[email protected]>
Reviewed-by: Brandon Nesterenko <[email protected]>
Oleksandr Byelkin
fix version
sjaakola
MDEV-41597: Do privilege checks of triggers and views before TOI

CREATE TRIGGER, CREATE VIEW and ALTER VIEW did some of their privilege
checks only after TOI had replicated the statement. The applier runs
with all privileges, so a statement refused on the originating node
was still applied by the other nodes:
- the DEFINER clause (sp_process_definer(), which requires SET USER
  for a DEFINER other than the current user), for all three
  statements, creating the trigger or view with a DEFINER the user
  was not allowed to use
- the column privileges of a view (ER_COLUMNACCESS_DENIED_ERROR),
  creating a view that gives more privileges than the user has on
  the underlying tables

ALTER VIEW without a DEFINER clause also replicated the wrong
definition. fill_defined_view_parts(), which takes DEFINER, ALGORITHM
and SQL SECURITY from the existing view, ran after the replicated
statement had been built, so the other nodes got the current user as
DEFINER and the default SQL SECURITY.

Do the checks before TOI begins, as is already done for stored
routines and events:
- mysql_create_or_drop_trigger() calls sp_process_definer() for
  CREATE TRIGGER before WSREP_TO_ISOLATION_BEGIN, and
  Table_triggers_list::create_trigger() no longer calls it.
- mysql_create_view() calls fill_defined_view_parts() (ALTER VIEW)
  and sp_process_definer() early, and begins TOI only after all
  checks, just before mysql_register_view().

The order of checks changes for all builds: the DEFINER error is now
reported before some errors that were found later in the statement.

The test galera.galera_toi_definer_check checks that:
- CREATE TRIGGER, CREATE VIEW and ALTER VIEW refused for a foreign
  DEFINER on node_1 are not applied on node_2
- CREATE VIEW refused for missing column privileges on node_1 is not
  applied on node_2
- ALTER VIEW without a DEFINER clause keeps the same DEFINER and
  SQL SECURITY on both nodes

Assisted By: Claude Opus 5.5
Marko Mäkelä
mysys patch adapted from Monty

my_copy(): Make use of copy_file_range(2) and block-cloning.

my_copy_file(): Copy between my_native_file handles.

my_copy_file_range(): Copy a range between my_native_file handles.

my_clone_file_range(): Clone a range between my_native_file handles
(Linux only; FreeBSD does this on my_copy_file_range()).

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

TODO: Replace sql_backup.cc with these
sjaakola
MDEV-41597: GRANT ROLE is replicated before the privilege check

GRANT ROLE and REVOKE ROLE started TOI before mysql_grant_role()
checked whether the user may grant the role, and they could be
replicated, although they potentially could be refused on the
originating node later on.
On the other nodes the applier runs with a non-user security context,
and GRANT and REVOKE ROLE, will be applied causing the nodes' ACLs
to be diverged.

Begin TOI is moved inside mysql_grant_role(), after the role lookup and the
privilege check. The check is done by a new helper,
find_grantable_role(), which is also used for the second check under
LOCK_grant.

Under TOI the second check only looks up the role again and does not
repeat the privilege check. Otherwise an ACL change ordered between
the first check and the start of TOI (e.g. REVOKE ADMIN OPTION from
the granting user) could refuse the statement on the originating node
only, while the other nodes apply it. The pre-TOI check is therefore
final in a cluster, as it already is for the other ACL statements:
privileges are checked as of the start of the statement. Without
wsrep, or outside TOI, both checks are done as before.

A DEBUG_SYNC point, grant_role_before_toi, is added before TOI begins.

The test galera.MDEV-41597 checks that:
- a GRANT ROLE refused on node_1 is not applied on node_2
- when REVOKE ADMIN OPTION is ordered between the privilege check and
  the start of TOI, both nodes reach the same outcome
Tarun Wadhwa
MDEV-37605: Extend mariadb-binlog to convert InnoDB-based binlogs to legacy format

MariaDB 12.3 added support for storing binary logs inside InnoDB
tablespace files (.ibb) instead of traditional flat sequential files.
There was previously no way to convert these back to the legacy
format, which is needed for downgrades, migrations, and third-party
tooling that only understands classic binlogs.

This adds a --convert-engine-binlog option to mariadb-binlog that
reads one or more InnoDB-based binlogs and writes out equivalent
legacy-format binlog files.

Usage:
  mariadb-binlog --convert-engine-binlog -r output_legacy_binlog \
      input1.ibb input2.ibb

This produces output_legacy_binlog.0000001, .0000002, etc. Note
there isn't a direct correlation between the number of input files
and output files. The output is split whenever the current file
hits --max-binlog-size (default 1GB, since legacy binlogs cannot
exceed 4GB), and also on every server restart recorded in the
InnoDB binlog, so a single input file can produce several output
files and a restart boundary can force a split even before the
size limit is reached.

Additionally, --max-binlog-size is also supported as a standalone
option to control this rotation threshold directly.

Only --convert-engine-binlog, -r, and --max-binlog-size are
supported alongside this mode; any other option will cause
mariadb-binlog to throw an error

Caveats:
- Encryption: InnoDB-based binlogs don't currently support
  encryption, so the converter doesn't handle it either. Output
  legacy binlogs are never encrypted by this mode.
- Checksums: InnoDB-based binlogs rely on page-level checksums
  instead of the legacy per-event checksums, so the initial
  implementation generates legacy output with no event-level
  checksum.

Reviewed-by: Kristian Nielsen <[email protected]>
Reviewed-by: Brandon Nesterenko <[email protected]>
Marko Mäkelä
Move some platform differentiation to backup_target member functions
Tarun Wadhwa
MDEV-37605: Extend mariadb-binlog to convert InnoDB-based binlogs to legacy format

MariaDB 12.3 added support for storing binary logs inside InnoDB
tablespace files (.ibb) instead of traditional flat sequential files.
There was previously no way to convert these back to the legacy
format, which is needed for downgrades, migrations, and third-party
tooling that only understands classic binlogs.

This adds a --convert-engine-binlog option to mariadb-binlog that
reads one or more InnoDB-based binlogs and writes out equivalent
legacy-format binlog files.

Usage:
  mariadb-binlog --convert-engine-binlog -r output_legacy_binlog \
      input1.ibb input2.ibb

This produces output_legacy_binlog.0000001, .0000002, etc. Note
there isn't a direct correlation between the number of input files
and output files. The output is split whenever the current file
hits --max-binlog-size (default 1GB, since legacy binlogs cannot
exceed 4GB), and also on every server restart recorded in the
InnoDB binlog, so a single input file can produce several output
files and a restart boundary can force a split even before the
size limit is reached.

Additionally, --max-binlog-size is also supported as a standalone
option to control this rotation threshold directly.

Only --convert-engine-binlog, -r, and --max-binlog-size are
supported alongside this mode; any other option will cause
mariadb-binlog to throw an error.

Caveats:
- Encryption: InnoDB-based binlogs don't currently support
  encryption, so the converter doesn't handle it either. Output
  legacy binlogs are never encrypted by this mode.
- Checksums: InnoDB-based binlogs rely on page-level checksums
  instead of the legacy per-event checksums, so the initial
  implementation generates legacy output with no event-level
  checksum.

Reviewed-by: Kristian Nielsen <[email protected]>
Reviewed-by: Brandon Nesterenko <[email protected]>
Daniel Bartholomew
bump the VERSION
Oleksandr Byelkin
fix version
Marko Mäkelä
fixup! e3994e5a387d754120584b1c52218c2f1374527c
Alexey Botchkov
How i see we can get rid of superfluous json manipulations.

We can make sure that the val_json() only returns valid JSONs.
Also we can add the 'nice' argument to the val_json() function
that would require the returning JSON be 'nice' as after the
json_nice(LOOSE) call.
Daniel Bartholomew
bump the VERSION
Marko Mäkelä
Move some platform differentiation to backup_target member functions
Marko Mäkelä
mysys patch adapted from Monty

my_copy(): Make use of copy_file_range(2) and block-cloning.

my_copy_file(): Copy between my_native_file handles.

my_copy_file_range(): Copy a range between my_native_file handles.

my_clone_file_range(): Clone a range between my_native_file handles
(Linux only; FreeBSD does this on my_copy_file_range()).

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

TODO: Replace sql_backup.cc with these
Marko Mäkelä
mysys patch adapted from Monty

my_copy(): Make use of copy_file_range(2) and block-cloning.

my_copy_file(): Copy between my_native_file handles.

my_copy_file_range(): Copy a range between my_native_file handles.

my_clone_file_range(): Clone a range between my_native_file handles
(Linux only; FreeBSD does this on my_copy_file_range()).

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

TODO: Use these in backup
Marko Mäkelä
mysys patch adapted from Monty

my_copy(): Make use of copy_file_range(2) and block-cloning.

my_copy_file(): Copy between my_native_file handles.

my_copy_file_range(): Copy a range between my_native_file handles.

my_clone_file_range(): Clone a range between my_native_file handles
(Linux only; FreeBSD does this on my_copy_file_range()).

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

TODO: Replace sql_backup.cc with these
Jan Lindström
MDEV-40150 galera tests for close() hang and mdev-36554 race

galera.mdev-36554: node_2's DELETE FROM t1 could race node_1's
replicating CREATE TABLE/INSERT and fail with ER_NO_SUCH_TABLE. Wait
for the row to arrive first. Require galera >= 26.4.30, which carries
the close()-vs-shift_to_CLOSED fix needed to run the rest of this test
safely.

galera.MDEV-40150: if the last receiver thread exits on an
inconsistency error while close() has already set closing_, the
provider never reaches CLOSED and close() waits forever. The test makes
the receiver exit with the inconsistency error, holds it before the
closing_ check, runs close() and then releases the receiver.

galera.MDEV-40150_vote: an applier waits for the result of an
inconsistency vote while the GCS receive thread exits on a core error.
Galera now wakes the voter, so the applier finds the vote failed and
the node leaves the cluster.

The two new tests need the debug points from Galera 26.4.30 or later
(MGL-359).

Assisted-by: Claude:claude-4.6-sonnet
Swayam-code
MDEV-34654: Use a hex literal for backslash in relaylog test

Replace char(92) with 0x5c in the path normalization expressions and
matching expected output, as suggested in review.

Assisted-by: Codex:GPT-6
Oleksandr Byelkin
fix version
sjaakola
MDEV-41597: GRANT ROLE is replicated before the privilege check

GRANT ROLE and REVOKE ROLE started TOI before mysql_grant_role()
checked whether the user may grant the role, and they could be
replicated, although they potentially could be refused on the
originating node later on.
On the other nodes the applier runs with a non-user security context,
and GRANT and REVOKE ROLE, will be applied causing the nodes' ACLs
to be diverged.

Begin TOI is moved inside mysql_grant_role(), after the role lookup and the
privilege check. The check is done by a new helper,
find_grantable_role(), which is also used for the second check under
LOCK_grant.

Under TOI the second check only looks up the role again and does not
repeat the privilege check. Otherwise an ACL change ordered between
the first check and the start of TOI (e.g. REVOKE ADMIN OPTION from
the granting user) could refuse the statement on the originating node
only, while the other nodes apply it. The pre-TOI check is therefore
final in a cluster, as it already is for the other ACL statements:
privileges are checked as of the start of the statement. Without
wsrep, or outside TOI, both checks are done as before.

A DEBUG_SYNC point, grant_role_before_toi, is added before TOI begins.

The test galera.MDEV-41597 checks that:
- a GRANT ROLE refused on node_1 is not applied on node_2
- when REVOKE ADMIN OPTION is ordered between the privilege check and
  the start of TOI, both nodes reach the same outcome
Swayam-code
MDEV-34654 perfschema.relaylog can fail based on the path of the build directory

Match and display log basenames after normalizing path separators, so
parent directory names cannot affect file selection or filename output.

Use misleading datadir names to exercise filtering, extraction and
relay-log rotation.

Assisted-by: Codex:GPT-6
Oleksandr Byelkin
fix version