Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Alessandro Vetere
MDEV-39600 Reduce contention in buf_flush_ahead()

When calling buf_flush_ahead() with furious=false, the function
updates the async target buf_flush_async_lsn and, if necessary notifies
the page cleaner thread to start flushing more eagerly, to avoid
a long stall later when checkpoint will be required.

When many threads call buf_flush_ahead() with furious=false, if
the update of buf_flush_async_lsn is gated inside
buf_pool.flush_list_mutex, this can cause significant contention,
just to deliver the new buf_flush_async_lsn value and a potential
notification to the page cleaner thread.

This commit enables, for the non furious case, lock-free update
of buf_flush_async_lsn and lock-free check of the page cleaner idle
flag, to avoid the contention on buf_pool.flush_list_mutex.
The lock is acquired by the CAS-loop winner thread and only
after the page cleaner is observed idle.
The cleaner clears buf_flush_async_lsn via CAS, so a concurrent
lock-free bump is preserved across the clear.
Since the reads are not protected by the mutex, there is a chance that
the decision to not signal the page cleaner thread is wrong.
In the worst case though, under sustained redo-log pressure,
eventually a wakeup signal will be delivered to the page cleaner thread
anyway, so the risk of a long stall is limited.

Similar reasoning applies to the possibly wrong decision taken by the
page cleaner thread to go to idle: eventually it will be woken up.
Vladislav Vaintroub
CI - Fix Github Actions warning, bump actions/checkout version
Vladislav Vaintroub
improve MTR bootstrap performance on slow disk

use --debug-no-sync to workaround slow disk flush.
On Github actions Windows ARM64, this alone cuts the test time from
ca 55 min to ca 45 min.
Dave Gosselin
MDEV-39746: FULL JOIN with a nested join on the right loses rows

The outermost FULL JOIN's right operand can be a nested join rather
than a single base table.  The parser places the nest on the right
when the outermost FULL JOIN's ON is the last one written, because the
parser keeps the outermost FULL JOIN pending until its ON arrives, and
the inner FULL JOINs reduce first into a nest that becomes the right
operand.
alloc_full_join_duplicate_filters allocates the fj_dups filter on a
JOIN_TAB carrying JOIN_TYPE_FULL | JOIN_TYPE_RIGHT, so with the
FULL|RIGHT bits on the nest, which is never a JOIN_TAB, no filter was
allocated and the null complement pass never fired.  The unmatched
rows from the right side were never emitted, producing a result with
missing rows.

Add swap_full_join_sides, called from rewrite_full_outer_joins
when a FULL JOIN survives simplify_joins with a leaf on the left
and a nested join on the right.  FULL JOIN is symmetric on its
operands, so swapping does not change query semantics; after the
swap the leaf carries the FULL|RIGHT bits and the rescan target
is a single base table.
Dmitry Shulga
MDEV-38561: ASAN heap-use-after-free in Query_arena::free_items/sp_lex_cursor::~sp_lex_cursor

On re-parsing of a failed cursor statement inside the stored routine,
the free_list of sp_lex_cursor could point to items placed on a dedicated
memory root that is created during re-parsing of the failed statement.

In result, when sp_head object is destroyed the mem_root created for
re-parsing the cursor's statement is de-allocated but the free_list
pointer of sp_lex_cursor still point to objects previously allocated
on this memory root.

To fix the issue, delay deallocation of mem_roots created for re-parsing
of SP instructions till the moment the sp_head be destroyed.
Dave Gosselin
Reject a nested join on the right of a rewritten FULL JOIN

check_full_join_base_tables runs before simplify_joins and rejects the
disallowed FULL JOIN shapes that are visible in the parse tree.
simplify_joins can rewrite a FULL JOIN to a LEFT, RIGHT, or INNER
JOIN, so sometimes disallowed queries appear only afterward.

Add check_full_join_after_simplify, called from optimize_inner once
simplify_joins is done, to reject unsupported queries after
rewrite by simplify_joins.
Dave Gosselin
MDEV-39014: FULL JOIN Phase 2

In phase 1, FULL [OUTER] JOIN was only supported when simplify_joins()
could rewrite it into an equivalent LEFT, RIGHT, or INNER JOIN based
on NULL-rejecting WHERE predicates.  Queries that could not be
rewritten raised ER_NOT_SUPPORTED_YET.  (Phase 1 was not released.)

This commit removes that restriction by adding proper support for FULL
JOIN by executing a 'LEFT JOIN pass' that emits matched rows and left
null-complemented rows, then a second "null-complement" pass which
rescans the right table to emit null-complement rows that were never
matched.

FULL JOIN supports nested joins on the left of the FULL JOIN,
NATURAL FULL JOIN, semi-joins, CTEs / derived tables (kept
materialized when they participate in a FULL JOIN), prepared
statements, stored procedures, and aggregates.  Examples:

  SELECT * FROM (d1 FULL JOIN d2 ON d1.a = d2.a)
              FULL JOIN t3 ON d1.a = t3.a;

  SELECT * FROM t1 NATURAL FULL JOIN t2;

  SELECT * FROM t1 INNER JOIN t2 FULL JOIN t3 ON t1.a = t3.a;

  PREPARE st FROM
    'SELECT COUNT(*) FROM t1 FULL JOIN t2 ON t1.a = t2.a';

Limitations:
  - Statistics and cost estimates for the null-complement pass have
    not been fully implemented; the optimizer may under- or
    over-estimate FULL JOIN costs in plans involving multiple
    FULL JOINs.  Again, a follow-up will optimize the cost calculations.
  - Optimizations for constant tables not fully supported.
  - Nested tables on the right side of a FULL JOIN are not yet supported.
PranavKTiwari
removed temp check.
Vladislav Vaintroub
MDEV-39708 invalid supplier name for MariaDB's own components

Fix sbom_get_supplier() to correctly return supplier name to caller
in all cases.
bsrikanth-mariadb
MDEV-39538: Different costs when same range is read twice

When same range is used as a filter, once in the outer query block, and
the second inside a sub query such as: -

select * from t1
  where year(a) = 2010 and c < (select count(*) from t1 where year(a) = 2010);

The Optimizer Context had two records for multi_range_read_info_const() call,
but had different cost vector members.
The cause is that the first table considered index-only scan on the range,
while the second considered non-index only scan.

However, when replaying the context, the same range got matched
twice, and the costs corresponding to that got returned twice.
Hence the costs in the explain plan output differed as well.

Solution
========
Include a new field called "call_number", while recording range contexts
into the overall context. This way, we could even match the call_number
and return the appropriate cost during replay.
Razvan Liviu Varzaru
MDBF-1214 Add WITH_{MSAN, ASAN, UBSAN} CMake support (#313)

* MDBF-1214 Add WITH_MSAN CMake support

Declare WITH_MSAN as a regular option for standalone Connector/C
builds.

When enabled directly in Connector/C, append MemorySanitizer compile and
link flags for Debug and RelWithDebInfo builds.

When Connector/C is included as a subproject, keep using the inherited
WITH_MSAN value only for sanitizer-specific linker behavior. This avoids
duplicating sanitizer flags already managed by parent projects.

* MDBF-1214 Add ASAN and UBSAN CMake support

Add standalone Connector/C options for WITH_ASAN and WITH_UBSAN.

Use a shared sanitizer flag helper for Debug and RelWithDebInfo builds,
covering compile flags and executable/shared/module linker flags.

Keep the flag handling disabled for subproject builds so Connector/C++
and ODBC continue to own sanitizer flags for their combined builds.

* MDBF-1214 Refine sanitizer CMake options

Add an opt-in WITH_ASAN_SCOPE option for Clang builds so ASAN can detect stack use after scope without adding the flag to linker options.

Enable MSVC ASAN stack use after return instrumentation and keep incremental linking disabled for those builds.

Reject MSAN when ASAN or UBSAN is also enabled. Remove the redundant ASAN -fPIC flag and keep UBSAN alignment checks enabled.
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB FTS

InnoDB FTS performed all reads and DML on its auxiliary,
common and CONFIG tables through the InnoDB internal SQL
graph parser. Replace that path with direct B-tree access
via a new query-executor layer, and delete the parser-era
helpers.

row0query.h, row/row0query.cc - new QueryExecutor:
- General MVCC-aware record traversal and basic DML on the
clustered index. Sits on a btr_pcur and a transaction-owned
mtr; record processing goes through a RecordCallback that
bundles two std::functions:
  compare_record() returns SKIP / PROCESS / STOP for a record
  process_record() handles each PROCESS-ed (MVCC-visible) row

Public API:
  read()              scan a clustered index with a search key
  read_all()          full clustered scan (optional start tuple)
  read_by_index()    scan a secondary index, fetch the matching
                      clustered record, deliver it to the callback
  insert_record()    insert a tuple into the clustered index
  delete_record()    delete a row identified by tuple
  delete_all()        delete every row in the clustered index
  select_for_update() position+X-lock the matching clustered row
  update_record()    update the row select_for_update() locked,
                      falling back to optimistic/pessimistic and
                      external storage paths as needed
  replace_record()    upsert: select_for_update()+update_record(),
                      else insert_record()
  lock_table(), handle_wait(), commit_mtr()

fts0exec.h, fts/fts0exec.cc - new FTSQueryExecutor:
Thin wrapper over QueryExecutor specialised for FTS tables.
Opens and locks the required tables once and exposes typed
helpers keyed by table family.

Auxiliary INDEX_[1..6]:
  open_all_aux_tables()
  insert_aux_record(aux_index, fts_aux_data_t)
  delete_aux_record(aux_index, fts_aux_data_t)
  read_aux()        range scan from a given word
  read_from_range()  paginated read that absorbs
                    DB_FTS_EXCEED_RESULT_CACHE_LIMIT internally
                    and resumes from the last word seen

Common deletion tables (DELETED, DELETED_CACHE, BEING_DELETED,
                        BEING_DELETED_CACHE):
  open_all_deletion_tables()
  insert_common_record(), delete_common_record(),
  delete_all_common_records(), read_all_common()

CONFIG table (<key, value>):
  open_config_table() / set_config_table()
  insert_config_record(), update_config_record() (upsert),
  delete_config_record(), read_config_with_lock()

fts_aux_data_t carries the auxiliary row payload.
RecordCallback specialisations live alongside the executor:

CommonTableReader collects doc_ids from common tables that
share the <doc_id> schema.

ConfigReader extracts <key, value> and provides
compare_config_key() for fast key matching.

AuxRecordReader scans auxiliary indexes with an
AuxCompareMode (GREATER_EQUAL / GREATER / LIKE / EQUAL) driving the
comparator; tracks the last word seen so a paginated scan can resume.

fts_query() walks index and common tables via
QueryExecutor::read_by_index() with RecordCallback;

fts_write_node() writes auxiliary rows through
FTSQueryExecutor::insert_aux_record() / delete_aux_record()
with fts_aux_data_t.

fts_optimize_write_word() now goes through the same
insert/delete path.

fts_select_index{,_by_range,_by_hash} return uint8_t (was
ulint) with a simpler control flow.

fts_optimize_table() binds a thd to its transaction whether
invoked from a user thread or the FTS optimize thread.

fts_optimize_t drops its fts_index_table and fts_common_table
fts_table_t fields; fts_query_t drops fts_common_table.

storage/innobase/fts/fts0sql.cc is deleted along with the
commented-out and unreferenced parser-era helpers it held.

dict_sys.latch is now acquired once per fts_sync_table(),
fts_optimize_table() and fts_query() call to open every
auxiliary and common table in one pass, instead of being
re-acquired per table.

Every FTS trx_create() site now takes the THD from a real
caller (user trx, fts_opt_thd, DDL ctx->trx, ha_thd())
instead of either leaving it NULL or pulling current_thd.

fts_commit(): Each fts_commit_table() used to create transaction
and commit, producing N undo segments, N read views and N group-commit
batches for N FTS tables touched by the user statement.
fts_commit() now creates one internal trx, runs every
fts_commit_table() under it, and commits once.

fts_optimize_word() could fail when source nodes have equal
boundary doc_ids.  Loosen the merge predicate from '>' to
'>=' so legitimate overlapping ranges that occur when nodes
fill to FTS_ILIST_MAX_SIZE at doc_id boundaries are handled
correctly.

INFORMATION_SCHEMA.INNODB_FT_INDEX_TABLE (i_s.cc) wires the
shared words_heap via set_words_heap() and empties it per batch
with mem_heap_empty(); the per-fetch cleanup frees only the
per-node ilist (ut_alloc'd) and skips fts_word_free().
Alexey Botchkov
MDEV-37262 XMLISVALID() schema validation function.

XMLVALID function added.
Sergei Petrunia
Fix ctype_uca.test: instead of SET NAMES use SET collation_connection
Alexey Botchkov
MDEV-37262 XMLISVALID() schema validation function.

XMLVALID function added.
Dave Gosselin
MDEV-38508: Constant table detection

If a table that's in a FULL OUTER JOIN is found to be a const
table, then don't allow the constant table optimization to
take place.

Later, when we support FULL OUTER JOIN on the inner side of
other join types then we may be able to relax this restriction.
Jan Lindström
MDEV-39798 : Galera test failure on galera_gcache_recover_manytrx

Problem was that file galera_wsrep_start_position.inc was not
found. Reason is still unknown. Added error handling
to galera_wsrep_recover.inc to find out the reason why
file is missing.
Dave Gosselin
MDEV-39569: Skip FULL JOIN rewrite to inner side of an outer join

Prevent simplify_joins from rewriting a chained FULL JOIN into a query
where a FULL JOIN could end up on the inner side of another outer
join.  Of course, this means that we will have a null complement pass
that the rewritten query would have avoided.  Once we support FULL
JOINs on the inner side of outer joins, in phase 3, then we can relax
this constraint.
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

In progress
Dave Gosselin
Update table_elim for FULL JOIN base table check

Two EXPLAIN queries in table_elim place a nested join on the right
side of a FULL JOIN.  Phase 2 supports only base tables there, so
check_full_join_base_tables rejects them with
ER_FULL_JOIN_BASE_TABLES_ONLY.
Daniel Bartholomew
bump the VERSION
bsrikanth-mariadb
MDEV-39791: Handle count aggregate optimization for replay purpose

Separately dump into the optimizer context, the count aggregation value
of any one non-nullable column of a table if any are used in the query.
This is only suppoerted for myisam tables.

During opt_sum_query() call, record the count aggregation values into
the optimizer context recorder. Once the context is being stored into
the information_schema table, dump all these recorded values into it.

During replay, parse the count aggregate values and store them in memory.
Later when opt_sum_query() requires the count value, hook the recorded
value from the parsed context.
Daniel Black
MDEV-37864 json_array_intersect result has incorrect length

MDEV-37864 added a length setting to the function
Item_func_json_array_intersect::prepare_json_and_create_hash
however some paths of Item_func_json_array_intersect::fix_length_and_dec
never reach this function.

Leave the length calculation the same but place in the
::fix_length_and_dec function.

Collation is based on the arg[0], but potentially after they
are swapped.

Corrects cursor protocol for the test case added in MDEV-36808.
Yuchen Pei
MDEV-39789 Fix compiling without perfschema
Marko Mäkelä
fixup! 7f207a291cb8b9903c08a05b0daa0a7eb90c3921
Daniel Bartholomew
bump the VERSION
Vladislav Vaintroub
MDEV-39719 Fix memory allocation errors on Windows ARM64 CI

Do not overload the machine. Huge reservation seem to stress
Windows on ARM64 memory management, so that random OOM can appear
when multiple processes are constantly allocating/deallocating
huge chunks of memory. Thus, reduce number of parallel threads to 2,
and do not run replication tests.
Daniel Bartholomew
bump the VERSION
bsrikanth-mariadb
MDEV-39791: Handle count aggregate optimization for replay purpose

Separately dump into the optimizer context, the count aggregation value
of any one non-nullable column of a table if any are used in the query.
This is only suppoerted for myisam tables.

During opt_sum_query() call, record the count aggregation values into
the optimizer context recorder. Once the context is being stored into
the information_schema table, dump all these recorded values into it.

During replay, parse the count aggregate values and store them in memory.
Later when opt_sum_query() requires the count value, hook the recorded
value from the parsed context.
Sergei Petrunia
Fix --replay-server error reporting: print query that caused error
Sergey Vojtovich
MDEV-21423 - lock-free trx_sys get performance regression cause by lf_find and ut_delay

Under high concurrency, MVCC snapshot creation may spend a significant
amount of time in lf_hash_iterate()/l_find() while collecting active
read-write transaction identifiers. This overhead is particularly
visible in sysbench oltp_read_write with
transaction-isolation=READ-COMMITTED.

Iteration cost becomes high due to significant TLB thrashing and poor
memory locality in this hot code path because snapshot creation touches
many rw_trx_hash nodes distributed across memory, including dummy
nodes that are irrelevant for snapshot construction. In addition,
traversing LF_HASH requires issuing heavyweight memory barriers.

This is a performance regression after 53cc9aa5be6, which changed
MVCC snapshot creation to scan LF_HASH instead of maintaining a global
sorted vector protected by the global mutex.

Add trx_sys.rw_trx_ids, a compact traversal-friendly vector of active
read-write transaction identifiers and serialization numbers optimized
for MVCC snapshot creation, while rw_trx_hash remains responsible for
transaction lookup.

The vector may contain empty slots corresponding to idle or read-only
transactions that currently do not own a read-write transaction
identifier. Such slots are skipped by snapshot creation.

This reduces traversal overhead during MVCC snapshot creation by
improving memory locality, reducing TLB pressure, and avoiding repeated
memory barriers required for rw_trx_hash traversal.
Oleksandr Byelkin
Merge branch '11.4' into 11.4 release
Vladislav Vaintroub
Merge 11.4 to 11.8