Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy

benchmarks.

To see the result in realistic load, run:
python run_mariadb_bench_inlined.py > results_final.plain
python convert_benchmarks.py
Nikita Malyavin
MDEV-12302 Move fk cascade depth control to THD

que_thr_t is not bypassed in between trigger calls, that is the count
starts from 0 each time in the trigger.

This means that an effective depth restriction is a multiplication of
FK_MAX_CASCADE_DEL and max_sp_recursion_depth.

Moving the control to THD changes this to addition.

Now the depth limit is: FK_MAX_CASCADE_DEL + max_sp_recursion_depth
Nikita Malyavin
MDEV-12302 Move fk cascade depth control to THD

que_thr_t is not bypassed in between trigger calls, that is the count
starts from 0 each time in the trigger.

This means that an effective depth restriction is a multiplication of
FK_MAX_CASCADE_DEL and max_sp_recursion_depth.

Moving the control to THD changes this to addition.

Now the depth limit is: FK_MAX_CASCADE_DEL + max_sp_recursion_depth
Nikita Malyavin
Fix cascade operations overwriting the record, when handler is re-used
in the nested row event.
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB FTS

InnoDB FTS performed all reads and DML on its auxiliary,
common and CONFIG tables through the InnoDB internal SQL
graph parser. Replace that path with direct B-tree access
via a new query-executor layer, and delete the parser-era
helpers.

row0query.h, row/row0query.cc - new QueryExecutor:
- General MVCC-aware record traversal and basic DML on the
clustered index. Sits on a btr_pcur and a transaction-owned
mtr; record processing goes through a RecordCallback that
bundles two std::functions:
  compare_record() returns SKIP / PROCESS / STOP for a record
  process_record() handles each PROCESS-ed (MVCC-visible) row

Public API:
  read()              scan a clustered index with a search key
  read_all()          full clustered scan (optional start tuple)
  read_by_index()    scan a secondary index, fetch the matching
                      clustered record, deliver it to the callback
  insert_record()    insert a tuple into the clustered index
  delete_record()    delete a row identified by tuple
  delete_all()        delete every row in the clustered index
  select_for_update() position+X-lock the matching clustered row
  update_record()    update the row select_for_update() locked,
                      falling back to optimistic/pessimistic and
                      external storage paths as needed
  replace_record()    upsert: select_for_update()+update_record(),
                      else insert_record()
  lock_table(), handle_wait(), commit_mtr()

fts0exec.h, fts/fts0exec.cc - new FTSQueryExecutor:
Thin wrapper over QueryExecutor specialised for FTS tables.
Opens and locks the required tables once and exposes typed
helpers keyed by table family.

Auxiliary INDEX_[1..6]:
  open_all_aux_tables()
  insert_aux_record(aux_index, fts_aux_data_t)
  delete_aux_record(aux_index, fts_aux_data_t)
  read_aux()        range scan from a given word
  read_from_range()  paginated read that absorbs
                    DB_FTS_EXCEED_RESULT_CACHE_LIMIT internally
                    and resumes from the last word seen

Common deletion tables (DELETED, DELETED_CACHE, BEING_DELETED,
                        BEING_DELETED_CACHE):
  open_all_deletion_tables()
  insert_common_record(), delete_common_record(),
  delete_all_common_records(), read_all_common()

CONFIG table (<key, value>):
  open_config_table() / set_config_table()
  insert_config_record(), update_config_record() (upsert),
  delete_config_record(), read_config_with_lock()

fts_aux_data_t carries the auxiliary row payload.
RecordCallback specialisations live alongside the executor:

CommonTableReader collects doc_ids from common tables that
share the <doc_id> schema.

ConfigReader extracts <key, value> and provides
compare_config_key() for fast key matching.

AuxRecordReader scans auxiliary indexes with an
AuxCompareMode (GREATER_EQUAL / GREATER / LIKE / EQUAL) driving the
comparator; tracks the last word seen so a paginated scan can resume.

fts_query() walks index and common tables via
QueryExecutor::read_by_index() with RecordCallback;

fts_write_node() writes auxiliary rows through
FTSQueryExecutor::insert_aux_record() / delete_aux_record()
with fts_aux_data_t.

fts_optimize_write_word() now goes through the same
insert/delete path.

fts_select_index{,_by_range,_by_hash} return uint8_t (was
ulint) with a simpler control flow.

fts_optimize_table() binds a thd to its transaction whether
invoked from a user thread or the FTS optimize thread.

fts_optimize_t drops its fts_index_table and fts_common_table
fts_table_t fields; fts_query_t drops fts_common_table.

storage/innobase/fts/fts0sql.cc is deleted along with the
commented-out and unreferenced parser-era helpers it held.

dict_sys.latch is now acquired once per fts_sync_table(),
fts_optimize_table() and fts_query() call to open every
auxiliary and common table in one pass, instead of being
re-acquired per table.

Every FTS trx_create() site now takes the THD from a real
caller (user trx, fts_opt_thd, DDL ctx->trx, ha_thd())
instead of either leaving it NULL or pulling current_thd.

fts_commit(): Each fts_commit_table() used to create transaction
and commit, producing N undo segments, N read views and N group-commit
batches for N FTS tables touched by the user statement.
fts_commit() now creates one internal trx, runs every
fts_commit_table() under it, and commits once.

fts_optimize_word() could fail when source nodes have equal
boundary doc_ids.  Loosen the merge predicate from '>' to
'>=' so legitimate overlapping ranges that occur when nodes
fill to FTS_ILIST_MAX_SIZE at doc_id boundaries are handled
correctly.

INFORMATION_SCHEMA.INNODB_FT_INDEX_TABLE (i_s.cc) wires the
shared words_heap via set_words_heap() and empties it per batch
with mem_heap_empty(); the per-fetch cleanup frees only the
per-node ilist (ut_alloc'd) and skips fts_word_free().
Nikita Malyavin
MDEV-12302 Move fk cascade depth control to THD

que_thr_t is not bypassed in between trigger calls, that is the count
starts from 0 each time in the trigger.

This means that an effective depth restriction is a multiplication of
FK_MAX_CASCADE_DEL and max_sp_recursion_depth.

Moving the control to THD changes this to addition.

Now the depth limit is: FK_MAX_CASCADE_DEL + max_sp_recursion_depth
Simoffsky
MDEV-12302 Execute triggers for foreign key updates/deletes
Sergei Petrunia
Rename list_ranges -> multi_range_read_info_const_calls
Nikita Malyavin
MDEV-31942 Online alter: support cascade foreign keys

All the technical provisioning was done yet by
MDEV-12302 Execute triggers for foreign key updates/deletes

As it executes ha_update_row and ha_delete_row directly.

All that is left is remove the restriction and add the tests.
Nikita Malyavin
MDEV-31942 Online alter: support cascade foreign keys

All the technical provisioning was done yet by
MDEV-12302 Execute triggers for foreign key updates/deletes

As it executes ha_update_row and ha_delete_row directly.

All that is left is remove the restriction and add the tests.
Sergei Petrunia
Rename list_records_in_range -> records_in_range_calls
Vladislav Vaintroub
MDEV-39951 DENY feature does not work for EVENT privilege

Fix calculation of need_table_or_column_check in check_access(),
so it does not consider already denied bits on DB level.

Add a test case for events (provided by Ramesh Sivaraman)
Sergei Petrunia
More include file renames.
Nikita Malyavin
MDEV-12302 Add SQL errors handling

Bypass the errors from sql layer, through innodb back to sql layer.
New handler error: HA_ERR_CASCADE_SQL
New innodb error: DB_SQL_ERROR

This error is not visible to user and serves only for correct error
transport. The real error code is expected to be stored in
thd->m_stmt_da, that is, already reported

This is a fixup commit.
Original author: Simoffsky <[email protected]>
Vladislav Vaintroub
MDEV-39951 DENY feature does not work for EVENT privilege

Fix calculation of need_table_or_column_check in check_access(),
so it does not consider already denied bits on DB level.

Add a test case for events (provided by Ramesh Sivaraman)
Nikita Malyavin
MDEV-12302 Refactor fk cascade depth check

* Move inc and dec for fk_cascade_depth close to each other, in a
symmetrical manner, wrapping the recursive call.

* Remove conputation-heavy node traversal with
row_ins_cascade_n_ancestors for each cascade call,
resulting in O(depth^2) total passes. It should be enough to do that
only as an assertion. Historically, there could be more than one thr,
joined in chains (but then we'd better pass over a chain of thr). Now a
single thr is used for cascade call, so no need in any graph traversal.

* Fix max cascade error handling -- now HA_ERR_FK_DEPTH_EXCEEDED is
returned, as implied.
Nikita Malyavin
MDEV-38840 add avx512 option
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy
Nikita Malyavin
Fix versioning.foreign crash

The problem is row_update_vers_insert invoked before the cascade update.
Cascade update relies on pre-defined node->pcur, and an insert can
invalidate the cursor, if b-tree node split happens.

One possible solution is to move the versioned insert after cascade
update.

This patch goes the described way, but also moves versioned insert to
the sql layer.

We have to re-set row event bits in trg_event_map, because
node->is_delete is determined based on those values.

TODO is to fix versioning.foreign 'trx_id,unique' result mismatch.
Sergei Petrunia
Rename include/get_rec_idx_ranges_from_opt_ctx.inc, cleaner printouts

New name: include/opt_context_list_tables_and_ranges.inc
Nikita Malyavin
MDEV-12302 Add SQL errors handling

Bypass the errors from sql layer, through innodb back to sql layer.
New handler error: HA_ERR_CASCADE_SQL
New innodb error: DB_SQL_ERROR

This error is not visible to user and serves only for correct error
transport. The real error code is expected to be stored in
thd->m_stmt_da, that is, already reported

This is a fixup commit.
Original author: Simoffsky <[email protected]>
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy

benchmarks.

To see the result in realistic load, run:
python run_mariadb_bench_inlined.py > results_final.plain
python convert_benchmarks.py
Nikita Malyavin
MDEV-31942 Online alter: support cascade foreign keys

All the technical provisioning was done yet by
MDEV-12302 Execute triggers for foreign key updates/deletes

As it executes ha_update_row and ha_delete_row directly.

All that is left is remove the restriction and add the tests.
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy
Dave Gosselin
MDEV-39914:  Crash in ST_SIMPLIFY used in an IN list

A geometry function writes its binary result into a buffer given by
the caller.  ST_SIMPLIFY did not set the charset of that buffer.  A
constant value in an IN list, though, is stored into the array that IN
builds for its comparisons, and that array has no charset.  Appending
the geometry then read an invalid charset and crashed the server.

Set the result buffer to the binary charset before writing the
geometry, matching what the other geometry functions already do.
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy

benchmarks.

To see the result in realistic load, run:
python run_mariadb_bench_inlined.py > results_final.plain
python convert_benchmarks.py
Nikita Malyavin
Fix versioning.foreign crash

The problem is row_update_vers_insert invoked before the cascade update.
Cascade update relies on pre-defined node->pcur, and an insert can
invalidate the cursor, if b-tree node split happens.

One possible solution is to move the versioned insert after cascade
update.

This patch goes the described way, but also moves versioned insert to
the sql layer.

We have to re-set row event bits in trg_event_map, because
node->is_delete is determined based on those values.

TODO is to fix versioning.foreign 'trx_id,unique' result mismatch.
Nikita Malyavin
Fix versioning.foreign crash

The problem is row_update_vers_insert invoked before the cascade update.
Cascade update relies on pre-defined node->pcur, and an insert can
invalidate the cursor, if b-tree node split happens.

One possible solution is to move the versioned insert after cascade
update.

This patch goes the described way, but also moves versioned insert to
the sql layer.

We have to re-set row event bits in trg_event_map, because
node->is_delete is determined based on those values.

TODO is to fix versioning.foreign 'trx_id,unique' result mismatch.
Nikita Malyavin
Fix cascade operations overwriting the record, when handler is re-used
in the nested row event.
Vladislav Vaintroub
MDEV-39951 DENY feature does not work for EVENT privilege

Fix calculation of need_table_or_column_check in check_access(), so it does
not include already denied bits.

Added a test case for events, provided by Ramesh Sivaraman
Sergei Petrunia
Rename: list_index_read_costs -> cost_for_index_read_calls
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy
Dave Gosselin
MDEV-39952:  Skip binlog_in_engine tests that need mariabackup

Several tests in the binlog_in_engine suite run the mariabackup
binary but did not check that it was built.  When the server is
built with WITH_MARIABACKUP=OFF, the test runner leaves the
mariabackup command empty and these tests fail instead of being
skipped.

Add include/have_mariabackup_binary.inc, which skips the test when
the binary is not available, and source it from the affected tests.
Unlike have_mariabackup.inc, it does not also require socat or nc,
because these tests copy the backup to a local directory rather than
streaming it.
Nikita Malyavin
Fix cascade operations overwriting the record, when handler is re-used
in the nested row event.
Thirunarayanan Balathandayuthapani
- Addressed the failures
Dave Gosselin
MDEV-36891:  std::make_pair compiler note

Construct the pair in place with emplace_back to silence the build
note.
Sergei Petrunia
Rename "list_contexts" to "tables"
Nikita Malyavin
MDEV-12302 Refactor fk cascade depth check

* Move inc and dec for fk_cascade_depth close to each other, in a
symmetrical manner, wrapping the recursive call.

* Remove conputation-heavy node traversal with
row_ins_cascade_n_ancestors for each cascade call,
resulting in O(depth^2) total passes. It should be enough to do that
only as an assertion. Historically, there could be more than one thr,
joined in chains (but then we'd better pass over a chain of thr). Now a
single thr is used for cascade call, so no need in any graph traversal.

* Fix max cascade error handling -- now HA_ERR_FK_DEPTH_EXCEEDED is
returned, as implied.
Dave Gosselin
MDEV-39952:  Skip tests that need mariabackup

Skips tests that require mariabackup if mariabackup was not
built (WITH_MARIABACKUP=OFF).

Backport of the same MTR change from 12.3 but applied to
additional tests.
Simoffsky
MDEV-12302 Execute triggers for foreign key updates/deletes