Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy

benchmarks.

To see the result in realistic load, run:
python run_mariadb_bench_inlined.py > results_final.plain
python convert_benchmarks.py
Nikita Malyavin
MDEV-12302 Move fk cascade depth control to THD

que_thr_t is not bypassed in between trigger calls, that is the count
starts from 0 each time in the trigger.

This means that an effective depth restriction is a multiplication of
FK_MAX_CASCADE_DEL and max_sp_recursion_depth.

Moving the control to THD changes this to addition.

Now the depth limit is: FK_MAX_CASCADE_DEL + max_sp_recursion_depth
Nikita Malyavin
Fix cascade operations overwriting the record, when handler is re-used
in the nested row event.
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB FTS

InnoDB FTS performed all reads and DML on its auxiliary,
common and CONFIG tables through the InnoDB internal SQL
graph parser. Replace that path with direct B-tree access
via a new query-executor layer, and delete the parser-era
helpers.

row0query.h, row/row0query.cc - new QueryExecutor:
- General MVCC-aware record traversal and basic DML on the
clustered index. Sits on a btr_pcur and a transaction-owned
mtr; record processing goes through a RecordCallback that
bundles two std::functions:
  compare_record() returns SKIP / PROCESS / STOP for a record
  process_record() handles each PROCESS-ed (MVCC-visible) row

Public API:
  read()              scan a clustered index with a search key
  read_all()          full clustered scan (optional start tuple)
  read_by_index()    scan a secondary index, fetch the matching
                      clustered record, deliver it to the callback
  insert_record()    insert a tuple into the clustered index
  delete_record()    delete a row identified by tuple
  delete_all()        delete every row in the clustered index
  select_for_update() position+X-lock the matching clustered row
  update_record()    update the row select_for_update() locked,
                      falling back to optimistic/pessimistic and
                      external storage paths as needed
  replace_record()    upsert: select_for_update()+update_record(),
                      else insert_record()
  lock_table(), handle_wait(), commit_mtr()

fts0exec.h, fts/fts0exec.cc - new FTSQueryExecutor:
Thin wrapper over QueryExecutor specialised for FTS tables.
Opens and locks the required tables once and exposes typed
helpers keyed by table family.

Auxiliary INDEX_[1..6]:
  open_all_aux_tables()
  insert_aux_record(aux_index, fts_aux_data_t)
  delete_aux_record(aux_index, fts_aux_data_t)
  read_aux()        range scan from a given word
  read_from_range()  paginated read that absorbs
                    DB_FTS_EXCEED_RESULT_CACHE_LIMIT internally
                    and resumes from the last word seen

Common deletion tables (DELETED, DELETED_CACHE, BEING_DELETED,
                        BEING_DELETED_CACHE):
  open_all_deletion_tables()
  insert_common_record(), delete_common_record(),
  delete_all_common_records(), read_all_common()

CONFIG table (<key, value>):
  open_config_table() / set_config_table()
  insert_config_record(), update_config_record() (upsert),
  delete_config_record(), read_config_with_lock()

fts_aux_data_t carries the auxiliary row payload.
RecordCallback specialisations live alongside the executor:

CommonTableReader collects doc_ids from common tables that
share the <doc_id> schema.

ConfigReader extracts <key, value> and provides
compare_config_key() for fast key matching.

AuxRecordReader scans auxiliary indexes with an
AuxCompareMode (GREATER_EQUAL / GREATER / LIKE / EQUAL) driving the
comparator; tracks the last word seen so a paginated scan can resume.

fts_query() walks index and common tables via
QueryExecutor::read_by_index() with RecordCallback;

fts_write_node() writes auxiliary rows through
FTSQueryExecutor::insert_aux_record() / delete_aux_record()
with fts_aux_data_t.

fts_optimize_write_word() now goes through the same
insert/delete path.

fts_select_index{,_by_range,_by_hash} return uint8_t (was
ulint) with a simpler control flow.

fts_optimize_table() binds a thd to its transaction whether
invoked from a user thread or the FTS optimize thread.

fts_optimize_t drops its fts_index_table and fts_common_table
fts_table_t fields; fts_query_t drops fts_common_table.

storage/innobase/fts/fts0sql.cc is deleted along with the
commented-out and unreferenced parser-era helpers it held.

dict_sys.latch is now acquired once per fts_sync_table(),
fts_optimize_table() and fts_query() call to open every
auxiliary and common table in one pass, instead of being
re-acquired per table.

Every FTS trx_create() site now takes the THD from a real
caller (user trx, fts_opt_thd, DDL ctx->trx, ha_thd())
instead of either leaving it NULL or pulling current_thd.

fts_commit(): Each fts_commit_table() used to create transaction
and commit, producing N undo segments, N read views and N group-commit
batches for N FTS tables touched by the user statement.
fts_commit() now creates one internal trx, runs every
fts_commit_table() under it, and commits once.

fts_optimize_word() could fail when source nodes have equal
boundary doc_ids.  Loosen the merge predicate from '>' to
'>=' so legitimate overlapping ranges that occur when nodes
fill to FTS_ILIST_MAX_SIZE at doc_id boundaries are handled
correctly.

INFORMATION_SCHEMA.INNODB_FT_INDEX_TABLE (i_s.cc) wires the
shared words_heap via set_words_heap() and empties it per batch
with mem_heap_empty(); the per-fetch cleanup frees only the
per-node ilist (ut_alloc'd) and skips fts_word_free().
Rex Johnston
MDEV-39859 alter parallel scan costs to favour putting table first.
Nikita Malyavin
MDEV-12302 Move fk cascade depth control to THD

que_thr_t is not bypassed in between trigger calls, that is the count
starts from 0 each time in the trigger.

This means that an effective depth restriction is a multiplication of
FK_MAX_CASCADE_DEL and max_sp_recursion_depth.

Moving the control to THD changes this to addition.

Now the depth limit is: FK_MAX_CASCADE_DEL + max_sp_recursion_depth
Sergei Petrunia
Rename list_ranges -> multi_range_read_info_const_calls
Nikita Malyavin
MDEV-31942 Online alter: support cascade foreign keys

All the technical provisioning was done yet by
MDEV-12302 Execute triggers for foreign key updates/deletes

As it executes ha_update_row and ha_delete_row directly.

All that is left is remove the restriction and add the tests.
Nikita Malyavin
MDEV-31942 Online alter: support cascade foreign keys

All the technical provisioning was done yet by
MDEV-12302 Execute triggers for foreign key updates/deletes

As it executes ha_update_row and ha_delete_row directly.

All that is left is remove the restriction and add the tests.
Sergei Petrunia
Rename list_records_in_range -> records_in_range_calls
Vladislav Vaintroub
MDEV-39951 DENY feature does not work for EVENT privilege

Fix calculation of need_table_or_column_check in check_access(),
so it does not consider already denied bits on DB level.

Add a test case for events (provided by Ramesh Sivaraman)
Sergei Petrunia
More include file renames.
Vladislav Vaintroub
MDEV-39951 DENY feature does not work for EVENT privilege

Fix calculation of need_table_or_column_check in check_access(),
so it does not consider already denied bits on DB level.

Add a test case for events (provided by Ramesh Sivaraman)
Yuchen Pei
[fixup] Move tests requiring an example plugin from sys_vars.session_track_system_variables_basic to a separate test

A follow up to 1f56d9c3feeeca82661cbe57cb628207c8b186f8. This restores
test coverage when the example plugin is not built.
Nikita Malyavin
MDEV-38840 add avx512 option
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy
Nikita Malyavin
Fix versioning.foreign crash

The problem is row_update_vers_insert invoked before the cascade update.
Cascade update relies on pre-defined node->pcur, and an insert can
invalidate the cursor, if b-tree node split happens.

One possible solution is to move the versioned insert after cascade
update.

This patch goes the described way, but also moves versioned insert to
the sql layer.

We have to re-set row event bits in trg_event_map, because
node->is_delete is determined based on those values.

TODO is to fix versioning.foreign 'trx_id,unique' result mismatch.
Sergei Petrunia
Rename include/get_rec_idx_ranges_from_opt_ctx.inc, cleaner printouts

New name: include/opt_context_list_tables_and_ranges.inc
Yuchen Pei
MDEV-39805 MDEV-39790 signal EOF in "unordered" partition index scans

This is a follow-up fix to MDEV-39535
96531691c18ebea4993454ee9d355e60ad9bea07 and MDEV-20195
8721a00dd38dc0aa1514a3b5ca8c95c6e94af1c9.

In those two fixes we added check for m_part_spec.start_part ==
NO_CURRENT_PART_ID (a typical sign of EOF) in
ha_partition::handle_unordered_prev and
ha_partition::handle_unordered_next respectively. However, because of
MDEV-37330 496aecf9251e3d74b83285f4e787daa9d8c37ae4, we skip a call to
handle_unordered_scan_next_partition inside these functions on
EOF (error == HA_ERR_END_OF_FILE and we have run out of partitions to
scan), whereas prior to MDEV-37330
496aecf9251e3d74b83285f4e787daa9d8c37ae4, it would call
handle_unordered_scan_next_partition which sets
m_part_spec.start_part= NO_CURRENT_PART_ID.

So in this patch we restore setting m_part_spec.start_part to
NO_CURRENT_PART_ID to signal EOF.
Nikita Malyavin
MDEV-12302 Add SQL errors handling

Bypass the errors from sql layer, through innodb back to sql layer.
New handler error: HA_ERR_CASCADE_SQL
New innodb error: DB_SQL_ERROR

This error is not visible to user and serves only for correct error
transport. The real error code is expected to be stored in
thd->m_stmt_da, that is, already reported

This is a fixup commit.
Original author: Simoffsky <[email protected]>
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy

benchmarks.

To see the result in realistic load, run:
python run_mariadb_bench_inlined.py > results_final.plain
python convert_benchmarks.py
Marko Mäkelä
fixup! f257584af9db2d2d435bdb2696515c54311663bc
Nikita Malyavin
MDEV-38840 optimize store_record/restore_record to use aligned copy
Dave Gosselin
MDEV-39914:  Crash in ST_SIMPLIFY used in an IN list

A geometry function writes its binary result into a buffer given by
the caller.  ST_SIMPLIFY did not set the charset of that buffer.  A
constant value in an IN list, though, is stored into the array that IN
builds for its comparisons, and that array has no charset.  Appending
the geometry then read an invalid charset and crashed the server.

Set the result buffer to the binary charset before writing the
geometry, matching what the other geometry functions already do.
Nikita Malyavin
Fix versioning.foreign crash

The problem is row_update_vers_insert invoked before the cascade update.
Cascade update relies on pre-defined node->pcur, and an insert can
invalidate the cursor, if b-tree node split happens.

One possible solution is to move the versioned insert after cascade
update.

This patch goes the described way, but also moves versioned insert to
the sql layer.

We have to re-set row event bits in trg_event_map, because
node->is_delete is determined based on those values.

TODO is to fix versioning.foreign 'trx_id,unique' result mismatch.
Daniel Black
BB can you give a s3 saved dir for MDEV-39829
Nikita Malyavin
Fix cascade operations overwriting the record, when handler is re-used
in the nested row event.
Vladislav Vaintroub
MDEV-39951 DENY feature does not work for EVENT privilege

Fix calculation of need_table_or_column_check in check_access(), so it does
not include already denied bits.

Added a test case for events, provided by Ramesh Sivaraman
Sergei Petrunia
Rename: list_index_read_costs -> cost_for_index_read_calls
Dave Gosselin
MDEV-39952:  Skip binlog_in_engine tests that need mariabackup

Several tests in the binlog_in_engine suite run the mariabackup
binary but did not check that it was built.  When the server is
built with WITH_MARIABACKUP=OFF, the test runner leaves the
mariabackup command empty and these tests fail instead of being
skipped.

Add include/have_mariabackup_binary.inc, which skips the test when
the binary is not available, and source it from the affected tests.
Unlike have_mariabackup.inc, it does not also require socat or nc,
because these tests copy the backup to a local directory rather than
streaming it.
Thirunarayanan Balathandayuthapani
- Addressed the failures
Dave Gosselin
MDEV-36891:  std::make_pair compiler note

Construct the pair in place with emplace_back to silence the build
note.
Thirunarayanan Balathandayuthapani
MDEV-39963  InnoDB system tablespace autoshrink fails when the tail extent is an empty XDES_FREE_FRAG extent

Problem:
========
- The InnoDB system tablespace fails to autoshrink even when it is
almost entirely free. Defragmentation reports success but no
space is reclaimed, and the log shows the high-water mark pinned
at the end of the file.

fsp_traverse_extents(): when locating the last used extent,
descends from the end of the tablespace and treats an extent
as reclaimable only when it is XDES_FREE, or the descriptor-page
extent (XDES_FREE_FRAG with two used pages).
Every other XDES_FREE_FRAG extent stops the scan.

An XDES_FREE_FRAG extent with zero used pages can legitimately exist
on disk: it is left behind when an emptied fragment extent is not
moved from FSP_FREE_FRAG to FSP_FREE (the condition fixed in
MDEV-31333) . Such an empty extent is logically identical to
XDES_FREE, but the traversal mistakes it for a used extent,
pins last_used_extent at the end of the file, and the shrink
reclaims nothing.

Solution:
========
fsp_traverse_extents(): Treat an XDES_FREE_FRAG extent with no used
pages (n_used == 0) the same as XDES_FREE
Dave Gosselin
MDEV-36891:  std::make_pair compiler note

Construct the pair in place with emplace_back to silence the build
note.
Sergei Petrunia
Rename "list_contexts" to "tables"
Nikita Malyavin
MDEV-12302 Refactor fk cascade depth check

* Move inc and dec for fk_cascade_depth close to each other, in a
symmetrical manner, wrapping the recursive call.

* Remove conputation-heavy node traversal with
row_ins_cascade_n_ancestors for each cascade call,
resulting in O(depth^2) total passes. It should be enough to do that
only as an assertion. Historically, there could be more than one thr,
joined in chains (but then we'd better pass over a chain of thr). Now a
single thr is used for cascade call, so no need in any graph traversal.

* Fix max cascade error handling -- now HA_ERR_FK_DEPTH_EXCEEDED is
returned, as implied.
Dave Gosselin
MDEV-39952:  Skip tests that need mariabackup

Skips tests that require mariabackup if mariabackup was not
built (WITH_MARIABACKUP=OFF).

Backport of the same MTR change from 12.3 but applied to
additional tests.
Alexander Barkov
MDEV-39587 Package-wide TYPE for variable declarations

SET sql_mode=ORACLE;
DELIMITER $$
CREATE OR REPLACE PACKAGE pkg AS
  -- Declare a package public data type
  TYPE varchar_array IS TABLE OF VARCHAR(2000) INDEX BY INTEGER;
END;
$$
DELIMITER ;
DELIMITER $$

CREATE OR REPLACE PROCEDURE p1 AS
  v pkg.varchar_array; -- Use the package public data type
BEGIN
  v(0):='test';
  SELECT v(0);
END;
$$
DELIMITER ;

Note, the change is done only for sql_mode=ORACLE, because the TYPE
declaration is not available for the default mode.

Where package-wide types are available
--------------------------------------
- Variabe list type:
    DECLARE var pkg1.type1;

- RETURN type for a package routine:
    CREATE FUNCTION .. RETURN pkg1.type1 ...

- Parameter type for a package routine:
    PROCEDURE p1(param1 pkg1.type1);

- Assoc array element type:
    TYPE assoc1_t IS TABLE OF pkg1.type1 ...

- REF CURSOR RETURN type:
    TYPE cur1_t IS REF CURSOR RETURN pkg1.type1;

Change details
--------------

- Adding a member Lex_length_and_dec_st::m_foreign_module_type
  It's set to true when the data type was initialized from a TYPE
  in foreign routine (e.g. in PACKAGE spec).
  It's needed to prevent use of qualified identifiers in public contexts,
  i.e. in schema public routine parameter types and schema publuc function
  RETURN types.
  Adding a helper method sp_head::check_applicability() which prevents
  use of qualified types in public context.

- Adding a helper method sp_head::raise_unknown_data_type().

- Adding methods LEX::set_field_type_typedef_package_spec() for
  2-step and 3-step qualified indentifiers.
  It's used in field_type_all_with_typedefs which covers cases:
  - Variabe list type        : DECLARE var pkg1.type1;
  - RETURN type              : CREATE FUNCTION .. RETURN pkg1.type1 ...
  - Parameter type          : PROCEDURE p1(param1 pkg1.type1);
  - Assoc array element type : TYPE assoc1_t IS TABLE OF pkg1.type1 ...

- Adding a method LEX::declare_type_ref_cursor_return_typedef().
  It handles cases when a new TYPE REF CURSOR RETURN is declared,
  for both for qualified RETURN types and non-qualified RETURN types:
  - TYPE cur0_t IS REF CURSOR RETURN rec1_t;
  - TYPE cur0_t IS REF CURSOR RETURN pkg1.rec1_t;
  - TYPE cur0_t IS REF CURSOR RETURN db1.pkg1.rec1_t;

  The code was moved from LEX::declare_type_ref_cursor() into
  LEX::declare_type_ref_cursor_return_typedef() and extended
  to cover qualified RETURN types.

- Adding a method Sql_path::find_package_spec_type().
  It iterates through all schemas specified in @@path and searches
  for the given type in the given package.

- Adding a helper method sp_pcontext::type_defs_add_ref_cursor()
  to reuse the code.

- Adding a new method sp_package::get_typedef() to search
  for TYPE definitions in PACKAGE specifications.

- Adding a new method sp_head::get_typedef_package_spec()
  to search for TYPE definitions used by a PROCEDURE or FUNCTION.

- Adding a helper method
    Sp_handler::sp_cache_routine_reentrant_suppress_errors
  Adding a method Sp_handler::find_package_spec().
Simoffsky
MDEV-12302 Execute triggers for foreign key updates/deletes