Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Rucha Deodhar
MDEV-40127: Server hangs when setting NEW=OLD on a multi-row table

Analysis:
m_fields was keeping old fields around across executions instead of
being cleared, causing stale state and hangs/crashes on re-execution.

Fix:
Clear m_fields in cleanup().
Rucha Deodhar
MDEV-40124: Assertion `m_sp == m_thd->spcont->m_sp' failed virtual
Item *Item_splocal::this_item(), UBSAN : member call on null pointer of
type 'Sp_rcontext_handler' in sql/item.cc

Analysis:
Comparing trigger row objects (IF NEW = OLD) calls cmp_row_type(),
which uses element_index(i). Item_trigger_row was missing its own
element_index() override, falling back to Item_splocal's version
and crashing/asserting on local context lookup.

Fix:
Override element_index(i) in Item_trigger_row to return
m_fields.elem(i) directly.
Kristian Nielsen
Fix hang on master when disabling semi-sync

There is a global variable global_ack_signal_fd used to signal the receiver
thread to wake up when disabling semi-sync. This variable was cleared to -1
in the Ack_listener destructor, which ran at the end of the
Ack_receiver::run() function without any locking. If the thread was delayed
at that point, it could end up overwriting the new value set by a new
receiver thread. This would leave the server in a state with an invalid
global_ack_signal_fd and could cause a subsequent disable of semisync to
fail due to the wakeup not arriving at the receiver thread.

This was seen as a sporadic failure of the test case
rpl.rpl_semi_sync_cond_var_per_thd.

Fix by not modifying the global in constructor/destructor; instead set and
clear the global explicitly, allowing to clear the fd with proper locking
while the mutex is still being held.

Also fix a missing pthread_join(), which would leak thread descriptors and
allow to start a new receiver thread before the old one shut down fully.

(Either of these two changes fix the hang bug).

Signed-off-by: Kristian Nielsen <[email protected]>
Monty
fixup! 18bf6b1b16f8c55b0d7d75ec860c0cf0fb88b7be
Marko Mäkelä
Suggested changes from Monty
Kristian Nielsen
Semi-sync: Implement support for using GTID in semi-sync ACKs

Implement the necessary logic in class Repl_semi_sync_master_gtid and
related code, so that the semi-sync master can request the slave to put the
GTID in the reply ACK packet, instead of the filename/offset.

In GTID mode, when a slave connects (which it should do using a GTID start
position), the latest GTID in the starting position (if any), as determined
by the list of transactions pending acks, is used as the point at which to
implicit ACK anything pending.

The semi-sync logic is otherwise unchanged, in GTID mode it just uses the
GTID as the transaction identifier instead of the file/pos of the end of the
event group.

This patch only enables the GTID-based semisync for binlog-in-engine, but
the mode works with old binlog implementation as well, and passes all tests
(if enabled by code change).

Signed-off-by: Kristian Nielsen <[email protected]>
Khaled Riyad
MDEV-40377 Change Server source code to point to new docs (11.4 part)

Replace the remaining Knowledge Base links with their MariaDB
Documentation equivalents, including the 838 URLs in the help tables.
Only URLs change in fill_help_tables.sql.

Merging upward: fill_help_tables.sql conflicts at 10.11->11.4 and
11.8->12.3. At those two merges keep the target branch's version,
since 11.4, 11.8 and 12.3 each carry their own URL fix. From 12.3 to
13.1 and 13.1 to main it merges cleanly; take the incoming change.
.github/pull_request_template.md is deleted in 12.3; keep the deletion.
Daniel Bartholomew
bump the VERSION
Oleksandr Byelkin
Fix memory leack in mysqltest
Yuchen Pei
MDEV-40168 [fixup] Fix some test failures

main.multi_valued_index_debug: different data dir in embedded
fixed msan
Kristian Nielsen
mysqltest: Implement --enable_sync_gtid option

The --enable_sync_gtid option switches to use GTID-based
--sync_slave_with_master (eg. using MASTER_GTID_WAIT() instead of
MASTER_POS_WAIT()).

This is useful to run adapt existing test cases for use with
--binlog-storage-engine. But it is also useful in general for replication
using GTID (which is the default).

The option is off by default to not randomly break existing tests.

Signed-off-by: Kristian Nielsen <[email protected]>
Kristian Nielsen
Implement support for semi-sync with --binlog-storage-engine

Call report_binlog_update() when semi-synchronous replication is enabled and
using --binlog-storage-engine.

Only AFTER_COMMIT is available. The AFTER_SYNC in legacy binlog has the
property that changes in a committing transaction does not get visible to
other transactions until the slave has acknowledged, avoiding phantom reads
if the master fails permanently just after. This requires the two-phase
commit between binlog and storage engine so that the binlog is written
before the transaction is engine-committed. However, the whole point of
--binlog-storage-engine is to avoid the expensive two-phase commit.

(The ability to avoid phantom reads could be later implemented as an
AFTER_PREPARE option, which would send binlog to slave and await ack before
it is written/committed into the engine).

Some existing semi-synchronous replication tests are adapted to also run in
the binlog_in_engine suite.

Signed-off-by: Kristian Nielsen <[email protected]>
Marko Mäkelä
MDEV-41152: Fix FILE_CREATE recovery

fil_name_process(): Treat FILE_CREATE in the same way as FILE_MODIFY
that led to a FIL_LOAD_DEFER return.
Khaled Riyad
MDEV-40377 Change Server source code to point to new docs (10.11 part)

Replace the remaining Knowledge Base links with their MariaDB
Documentation equivalents, and fix the 14 help table URLs pointing at
/README pages that do not exist. Only URLs change in
fill_help_tables.sql.

Merging upward: fill_help_tables.sql conflicts at 10.11->11.4 and
11.8->12.3. At those two merges keep the target branch's version,
since 11.4, 11.8 and 12.3 each carry their own URL fix. From 12.3 to
13.1 and 13.1 to main it merges cleanly; take the incoming change.
.github/pull_request_template.md is deleted in 12.3; keep the deletion.
Oleksandr Byelkin
fix typo
Monty
Added new mysys functions my_open_dir and improved my_copy

- Improved my_copy() using copy_file_range and memmap

- New mysys functions:
int my_copy_file(File from, File to, myf MyFlags);
int my_copy_file_range(File from, File to, my_off_t start,
                        my_off_t end, myf MyFlags);

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

Other things
- Fixed #ifdef's in sql_backup.cc to use the new define
  HAVE_COPY_FILE_RANGE
Rucha Deodhar
MDEV-34723: NEW and OLD in a trigger as row variables

Implementation:
NEW and OLD represent the entire table row. So it can be thought of as
list of Item_trigger_field. When we are in a trigger and NEW or OLD is
encountered, create Item_trigger_row object with same constructor as
Item_trigger_field, it will also be used later while creating
Item_trigger_field objects. Populate the m_fields list while
fixing fields. Create a corresponding instruction sp_instr_set_trigger_row
which will be used to set the values
Marko Mäkelä
MDEV-41152: Fix FILE_CREATE recovery

fil_name_process(): Treat FILE_CREATE in the same way as FILE_MODIFY
that led to a FIL_LOAD_DEFER return.

(cherry picked from commit a898d2b1ae1719fdbd11c4221b9a9ae8ec5334a6)
Rucha Deodhar
MDEV-40478: main.func_json_unicode_escape test unpredicatable under
view protocol

Analysis:
The function with arguments is too long for view column name. So view
protocol just renames it.
Fix:
Use alias to avoid renaming.
Monty
fixup! 18bf6b1b16f8c55b0d7d75ec860c0cf0fb88b7be
Kristian Nielsen
Semi-sync: Some few after-review fixes

Signed-off-by: Kristian Nielsen <[email protected]>
Monty
fixup! 5bd9778829a151239c8b96136bc05e5af6652b22
Kristian Nielsen
Semi-sync: Refactor in preparation for using GTID in semi-sync acks

This is a refactor patch that contains no/little logic changes but a lot of
mostly mechanic code changes to prepare for allowing to use either old-style
filename/offset or new-style GTID to identify an event group in the
semi-sync ack.

The idea is to replace all explicit filename/offset function arguments with
a generic Repl_semi_sync_trx_info *inf to identify an event group (aka
"transaction"). This object can then be used to look up in the semi-sync
hash table by either file/pos or by GTID.

The classes Active_tranx and Repl_semi_sync_master are sub-classed into
Active_tranx_file_pos/Active_tranx_gtid and
Repl_semi_sync_master_file_pos/Repl_semi_sync_master_gtid. Virtual functions
are implemented in each for comparing identifiers and for calculating hash
keys, using either the file/pos or the GTID as appropriate. This way, the
existing logic can now be used with either (only file/pos is actually used
in this patch; adding GTID is for a subsequent patch).

For the binlog writing side, report_binlog_update(), wait_after_sync(), and
THD::semisync_info are extended to also take the GTID of the event group,
which is already available in the calling code.

For the dump thread / slave connection side, update_sync_header() is
extended to take also the GTID. The dump thread code is extended to keep
track of the GTID of the current event group (slightly extending the logic
already there to keep track of event groups). Also, the dump thread now only
passes the last event of an event group into the semisync layer (the other
events are redundant, as they are never semi-sync ack'ed, saving needless
semisync locking and hash lookups).

Signed-off-by: Kristian Nielsen <[email protected]>
Rucha Deodhar
MDEV-34723: NEW and OLD in a trigger as row variables

Implementation:
NEW and OLD represent the entire table row. So it can be thought of as
list of Item_trigger_field. When we are in a trigger and NEW or OLD is
encountered, create Item_trigger_row object with same constructor as
Item_trigger_field, it will also be used later while creating
Item_trigger_field objects. Populate the m_fields list while
fixing fields. Create a corresponding instruction sp_instr_set_trigger_row
which will be used to set the values
Yuchen Pei
MDEV-39525 [wip] Add supertype checks to vcol index substitution in WHERE

- check if both the vcol expr and vcol field are supertypes to the RHS
- check if vcol field is a supertype to vcol expr

If either check returns true, then the substitution is safe.

Skip the checks for IS NULL / IS NOT NULL.

TODO: Does not yet work for the JSON_EXTRACT/JSON_VALUE vcol examples
in vcol_sargable because capacity_limit_is_in_characters returns
different values for varchar and blob/text
Rucha Deodhar
MDEV-40124: Assertion `m_sp == m_thd->spcont->m_sp' failed virtual
Item *Item_splocal::this_item(), UBSAN : member call on null pointer of
type 'Sp_rcontext_handler' in sql/item.cc

Analysis:
Comparing trigger row objects (IF NEW = OLD) calls cmp_row_type(),
which uses element_index(i). Item_trigger_row was missing its own
element_index() override, falling back to Item_splocal's version
and crashing/asserting on local context lookup.

Fix:
Override element_index(i) in Item_trigger_row to return
m_fields.elem(i) directly.
Rex Johnston
MDEV-38801  implement Item_cache_year shallow_copy()

Implement shallow_copy() for Item_cache_year, same as for Item_cache_bool
to avoid typeid mismatch from inheriting this method from Item_cache_int.
Triggered when creating a clone of a condition for pushdown into a
derived table.

(Testcase amended by Sergei Petrunia)
Monty
Added new mysys functions my_open_dir and improved my_copy

- Improved my_copy() using copy_file_range and memmap

- New mysys functions:
int my_copy_file(File from, File to, myf MyFlags);
int my_copy_file_range(File from, File to, my_off_t start,
                        my_off_t end, myf MyFlags);

- New functions for looping over files in a directory:
MY_NO_CACHE_DIR *my_dir_open();
int my_dir_read_next()
int my_dir_rewind();
int my_dir_close();

Other things
- Fixed #ifdef's in sql_backup.cc to use the new define
  HAVE_COPY_FILE_RANGE
Rucha Deodhar
MDEV-40124: Assertion `m_sp == m_thd->spcont->m_sp' failed virtual
Item *Item_splocal::this_item(), UBSAN : member call on null pointer of
type 'Sp_rcontext_handler' in sql/item.cc

Analysis:
Comparing trigger row objects (IF NEW = OLD) calls cmp_row_type(),
which uses element_index(i). Item_trigger_row was missing its own
element_index() override, falling back to Item_splocal's version
and crashing/asserting on local context lookup.

Fix:
Override element_index(i) in Item_trigger_row to return
m_fields.elem(i) directly.
Khaled Riyad
MDEV-40377 Change Server source code to point to new docs (11.8 part)

Replace the remaining Knowledge Base links with their MariaDB
Documentation equivalents, including the 838 URLs in the help tables.
Only URLs change in fill_help_tables.sql.

Merging upward: fill_help_tables.sql conflicts at 10.11->11.4 and
11.8->12.3. At those two merges keep the target branch's version,
since 11.4, 11.8 and 12.3 each carry their own URL fix. From 12.3 to
13.1 and 13.1 to main it merges cleanly; take the incoming change.
.github/pull_request_template.md is deleted in 12.3; keep the deletion.
Oleksandr Byelkin
Fix compiler pronlems
Monty
fixup! 5bd9778829a151239c8b96136bc05e5af6652b22
Kristian Nielsen
Semi-sync: Clean up incorrect file/pos comparisons

The semi-sync code has a number of places where it compares pairs of
(filename,offset) for which is larger than the other.

The filename comparisons are done using strcmp(), which is wrong. Filenames
will compare wrong when they wrap from eg. bin-999999 to bin-1000000, and
user can also rename log files which can likewise break comparisons.

Further, the comparisons are completely unnecessary, as all the transactions
to be waited for are already stored in a linear list in order _and_ in a
hash table. So the code can simply use the existing hash table look and list
traversal to determine status and sequence of the waited-for transactions.

So this patch removes all the comparisons for larger/smaller, leaving only
comparisons for equality and effectively making the (filename,offset) pairs
just opaque transaction identifiers. And also removes a few other related
pieces of dead/unnecessary code.

Signed-off-by: Kristian Nielsen <[email protected]>
Kristian Nielsen
Fix typo in comment

Signed-off-by: Kristian Nielsen <[email protected]>
Marko Mäkelä
WIP: log tracking BACKUP SERVER TO ... CONCURRENT (for HAVE_INNODB_PMEM)

backup_sink::id: The thread identifier (0 to CONCURRENT-1)

innodb_backup_checkpoint_pmem(): Copy the old log file.

InnoDB_backup::log_track(), InnoDB_backup::log_track_pmem():
Keep copying the log until we run out of InnoDB data files to copy.

InnoDB_backup::checkpoint_complete_pmem(): Copy the remaining
part of an old log file right before it is being released.

InnoDB_backup::commit(): In log tracking backup, copy the rest of
the HAVE_INNODB_PMEM log.

FIXME: Implement the non-PMEM code path with minimal blocking.
Monty
Integrate the old and new backup vofr

Fixes a lot of issues in current backup:
- Galera
- Enables ddl logging (so we can use it in the future)
- Flushes binary logs (we still must add code to copy them)
- mdl locks are consistent between maria-backup and backup command
- startup backup code for innodb moved to innodb_prepare_for_backup() called by prepare_for_backup hton handler.
- Give errors if backup command is done under a transaction, global read lock or lock tables.
- Retry for MDL_BACKUP_WAIT_DDL (needed for backup.backup_ddl_concurrent_verify)
- Removed wrong log locks in Aria

Things to do (in addition to the things in my review) :

At backup_stage stage start, force rotate of aria log files. This allows us to copy all old logs without any locks
Copy all transactional tables and old aria logs under BACKUP_START (as maria-backup does)
Copy the active aria log file under block commit.
Improve speed of copying aria tables by copy files in up to 1M blocks and run checksum on the blocks and only re-read blocks with wrong checksum.
Copy non transactional files under BACKUP_PHASE_NO_BEGIN_NON_TRANS.
 Note that Aria does not support the documented BACKUP_PHASE_NO_DML_NON_TRANS . The BACKUP_PHASE_NO_BEGIN_NON_TRANS state is already blocking changes to non transactional tables
Yuchen Pei
MDEV-40168 [fixup] Fix some test failures

main.multi_valued_index_debug: different data dir in embedded
fixed msan
Monty
Update backup code to use new my_dir and my_copy interfaces
Rucha Deodhar
MDEV-40127: Server hangs when setting NEW=OLD on a multi-row table

Analysis:
m_fields was keeping old fields around across executions instead of
being cleared, causing stale state and hangs/crashes on re-execution.

Fix:
Clear m_fields in cleanup().
Rucha Deodhar
MDEV-40127: Server hangs when setting NEW=OLD on a multi-row table

Analysis:
m_fields was keeping old fields around across executions instead of
being cleared, causing stale state and hangs/crashes on re-execution.

Fix:
Clear m_fields in cleanup().