Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
cleanup: spaces, formatting
Sergei Golubchik
w 6.4
Sergei Golubchik
w (rename, fix compilation)
Sergei Golubchik
temporarily remove failing tests
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

Refactoring according to Serg's review. In this version, SE/server API now
narrows the SE role to just report the changes done by foreign key cascading,
and server side does most of the work after that.

Added a design document MDEV-38243-design.md
Sergei Golubchik
w 6.0
Aleksey Midenkov
MDEV-41050 Aria/MyISAM: versioned DELETE via row_end index leaves a row undeleted

A system-versioned DELETE on a MyISAM or Aria table indexed by row_end
could leave the last current row undeleted. The delete scans the current
rows via an equality search on row_end = MAX, which is driven by
mi_rnext_same/maria_rnext_same. That function keeps the search's
reference key in lastkey2 and uses the HA_STATE_RNEXT_SAME flag to
remember it has already stored it. This reference-key mechanism is
exactly what lets the scan modify rows it is walking without skipping
them.

Deleting a versioned row is an in-place update of row_end, and the update
reuses lastkey2 as scratch space for the changed key, so it clears
HA_STATE_RNEXT_SAME to request that rnext_same re-store its reference on
the next call. However, TABLE::delete_row wraps the update in
HA_EXTRA_REMEMBER_POS/HA_EXTRA_RESTORE_POS, and RESTORE_POS restored the
whole saved info->update word, resurrecting the HA_STATE_RNEXT_SAME bit
that the update had just cleared.

As a result rnext_same skipped rebuilding its reference key and compared
subsequent keys against the now-overwritten lastkey2, hitting a spurious
end-of-file and terminating the scan one row early, defeating the engine's
own protection against a Halloween-style skip.

Fixed by preserving the current HA_STATE_RNEXT_SAME bit across
RESTORE_POS instead of restoring the stale saved value.
Sergei Golubchik
fix fulltext.test to break ties (rows with the same relevance)
Sergei Golubchik
cleanup: remove redundant function
Sergei Golubchik
cleanup: remove obsolete code
Sergei Golubchik
w 5

fts using hlindex class
Sergei Golubchik
cleanup: KEY::type() and KEY::is_hlindex() methods
Sergei Golubchik
w 6.1
Sergei Golubchik
w 6.7
Aleksey Midenkov
MDEV-41050 HEAP: versioned DELETE via row_end index leaves a row undeleted

The row loss also reproduces on the MEMORY (HEAP) engine. A system-
versioned DELETE scans the current rows on the row_end index and turns
each delete into an in-place update of row_end, so it modifies the very
index it is walking.

When the changed key is the scanned one (info->lastinx), hp_delete_key()
repositions the cursor but heap_update() leaves info->update untouched, so
HA_STATE_NEXT_FOUND from the preceding heap_rnext() stays set. The next
heap_rnext() then sees current_ptr == 0 with that bit and takes the
"!current_ptr && HA_STATE_NEXT_FOUND" guard as a false end-of-file,
stopping one row early.

Fixed by clearing HA_STATE_NEXT_FOUND when the scanned index key changed.
HA_STATE_AKTIV is kept (unlike heap_delete): the row is updated, not
removed, so a following op must not fail test_active(). The bit is only
set after a heap_rnext(), so a plain single-row UPDATE never reaches this.
Aleksey Midenkov
MDEV-41050 HEAP: versioned DELETE via row_end index leaves a row undeleted

The row loss also reproduces on the MEMORY (HEAP) engine. A system-
versioned DELETE scans the current rows on the row_end index and turns
each delete into an in-place update of row_end, so it modifies the very
index it is walking.

When the changed key is the scanned one (info->lastinx), hp_delete_key()
repositions the cursor but heap_update() leaves info->update untouched, so
HA_STATE_NEXT_FOUND from the preceding heap_rnext() stays set. The next
heap_rnext() then sees current_ptr == 0 with that bit and takes the
"!current_ptr && HA_STATE_NEXT_FOUND" guard as a false end-of-file,
stopping one row early.

Fixed by clearing HA_STATE_NEXT_FOUND when the scanned index key changed.
HA_STATE_AKTIV is kept (unlike heap_delete): the row is updated, not
removed, so a following op must not fail test_active(). The bit is only
set after a heap_rnext(), so a plain single-row UPDATE never reaches this.
Sergei Golubchik
w 4

* class hlindex and hlinex_share (sic!)
* mhnsw_index inherits from hlindex
* replace mhnsw* functions with mhnsw_index methods
* fix all callers in sql/* to use hlindex methods
Aleksey Midenkov
MDEV-41050 HEAP: versioned DELETE via row_end index leaves a row undeleted

The row loss also reproduces on the MEMORY (HEAP) engine. A system-
versioned DELETE scans the current rows on the row_end index and turns
each delete into an in-place update of row_end, so it modifies the very
index it is walking.

When the changed key is the scanned one (info->lastinx), hp_delete_key()
repositions the cursor but heap_update() leaves info->update untouched, so
HA_STATE_NEXT_FOUND from the preceding heap_rnext() stays set. The next
heap_rnext() then sees current_ptr == 0 with that bit and takes the
"!current_ptr && HA_STATE_NEXT_FOUND" guard as a false end-of-file,
stopping one row early.

Fixed by clearing HA_STATE_NEXT_FOUND when the scanned index key changed.
HA_STATE_AKTIV is kept (unlike heap_delete): the row is updated, not
removed, so a following op must not fail test_active(). The bit is only
set after a heap_rnext(), so a plain single-row UPDATE never reaches this.
Aleksey Midenkov
Fix comment reference in ha_extra.c

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Sergei Golubchik
w1

* move vector_hnsw.h to its new home sql/index

* new files index/hlindex.h and index/fts.cc
* new struct hlindexton
* old FTS is selected with FULLTEXT (...) USING ENGINE (questionable)
* KEY::options() method

* FULLTEXT_USING_ENGINE old mode
* FULLTEXT (...) USING TABLE to select new implementation
Sergei Golubchik
w 6.6 EXPLAIN
Sergei Golubchik
cleanup: const void* in TREE
Aleksey Midenkov
MDEV-41050 Aria/MyISAM: versioned DELETE via row_end index leaves a row undeleted

A system-versioned DELETE on a MyISAM or Aria table indexed by row_end
could leave the last current row undeleted. The delete scans the current
rows via an equality search on row_end = MAX, which is driven by
mi_rnext_same/maria_rnext_same. That function keeps the search's
reference key in lastkey2 and uses the HA_STATE_RNEXT_SAME flag to
remember it has already stored it. This reference-key mechanism is
exactly what lets the scan modify rows it is walking without skipping
them.

Deleting a versioned row is an in-place update of row_end, and the update
reuses lastkey2 as scratch space for the changed key, so it clears
HA_STATE_RNEXT_SAME to request that rnext_same re-store its reference on
the next call. However, TABLE::delete_row wraps the update in
HA_EXTRA_REMEMBER_POS/HA_EXTRA_RESTORE_POS, and RESTORE_POS restored the
whole saved info->update word, resurrecting the HA_STATE_RNEXT_SAME bit
that the update had just cleared.

As a result rnext_same skipped rebuilding its reference key and compared
subsequent keys against the now-overwritten lastkey2, hitting a spurious
end-of-file and terminating the scan one row early, defeating the engine's
own protection against a Halloween-style skip.

Fixed by preserving the current HA_STATE_RNEXT_SAME bit across
RESTORE_POS instead of restoring the stale saved value.
Sergei Golubchik
Tree<> - a typesafe wrapper for TREE
Vladislav Vaintroub
MDEV-38918 Make large pages an explicit per-caller opt-in

my_large_malloc() attempted large pages whenever --large-pages was
enabled, silently rounding the size up and reporting it back via an
in/out parameter. ut_malloc_dontdump() never passed that adjusted
size on to its own callers (the InnoDB redo log buffer and
recv_sys_t::tmp_buf), so freeing later used the original, smaller
size, causing the reported "faux memory leak".

Only the buffer pool and the MyISAM/Aria key caches are documented
to benefit from large pages. Everything else that ended up calling
my_large_malloc() only wanted its "do not dump to core" property and
picked up large pages as an undocumented side effect; those buffers
are also small and sequentially accessed, so they would have gained
little from large pages anyway.

Add MY_TRY_LARGE_PAGES: my_large_malloc() and my_large_virtual_alloc()
now only attempt large pages when a caller passes this flag, instead
of always trying whenever the global option is set. Only the buffer
pool and the key caches pass it. The redo log buffer, tmp_buf, and
row0log.cc's crypt buffers no longer request large pages at all,
which removes the size-rounding bug for them without touching that
code.

my_large_virtual_alloc()'s fallback (no usable large page size) must
also return read-write memory right away, like the Windows large-pages
fallback already does, since my_virtual_mem_commit() is a no-op for
MY_TRY_LARGE_PAGES. my_large_pages_flag is now set once, in
my_init_large_pages(), and never changed thereafter, on any platform.

Both my_virtual_mem_commit() and my_virtual_mem_decommit() are now
no-ops, aside from accounting, whenever large pages are requested.

Also fix a broken mtr suppression regex in main.large_pages that
would fail the test on Windows.
Sergei Golubchik
w 6.3
Sergei Golubchik
w
Sergei Golubchik
w
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

3'rd batch of review changes. Now the SE has to call for cascade service only
to report of row before and after image and the end of cascade operation.
There is new SE service interface: include/mysql/service_thd_fk_cascade.h
For server, there are 3 consumers for cascade operationss: binlogging,
firing triggers, client table FK checks, of which only binlogging has
implemeentation
Sergei Golubchik
cleanup: FT_INFO, please -> handler.h

Replace 26-year-old C-and-preprocesor-tricks FT_INFO API with
C++ classes and inheritance, as all engines are C++ anyway.

Rename classes and variables accordingly.
Aleksey Midenkov
MDEV-41050 Aria/MyISAM: versioned DELETE via row_end index leaves a row undeleted

A system-versioned DELETE on a MyISAM or Aria table indexed by row_end
could leave the last current row undeleted. The delete scans the current
rows via an equality search on row_end = MAX, which is driven by
mi_rnext_same/maria_rnext_same. That function keeps the search's
reference key in lastkey2 and uses the HA_STATE_RNEXT_SAME flag to
remember it has already stored it. This reference-key mechanism is
exactly what lets the scan modify rows it is walking without skipping
them.

Deleting a versioned row is an in-place update of row_end, and the update
reuses lastkey2 as scratch space for the changed key, so it clears
HA_STATE_RNEXT_SAME to request that rnext_same re-store its reference on
the next call. However, TABLE::delete_row wraps the update in
HA_EXTRA_REMEMBER_POS/HA_EXTRA_RESTORE_POS, and RESTORE_POS restored the
whole saved info->update word, resurrecting the HA_STATE_RNEXT_SAME bit
that the update had just cleared.

As a result rnext_same skipped rebuilding its reference key and compared
subsequent keys against the now-overwritten lastkey2, hitting a spurious
end-of-file and terminating the scan one row early, defeating the engine's
own protection against a Halloween-style skip.

Fixed by preserving the current HA_STATE_RNEXT_SAME bit across
RESTORE_POS instead of restoring the stale saved value.
Sergei Golubchik
cleanup: change mhnsw* code to use init/next api not first/next
Aleksey Midenkov
MDEV-41050 HEAP: versioned DELETE via row_end index leaves a row undeleted

The row loss also reproduces on the MEMORY (HEAP) engine. A system-
versioned DELETE scans the current rows on the row_end index and turns
each delete into an in-place update of row_end, so it modifies the very
index it is walking.

When the changed key is the scanned one (info->lastinx), hp_delete_key()
repositions the cursor but heap_update() leaves info->update untouched, so
HA_STATE_NEXT_FOUND from the preceding heap_rnext() stays set. The next
heap_rnext() then sees current_ptr == 0 with that bit and takes the
"!current_ptr && HA_STATE_NEXT_FOUND" guard as a false end-of-file,
stopping one row early.

Fixed by clearing HA_STATE_NEXT_FOUND when the scanned index key changed.
HA_STATE_AKTIV is kept (unlike heap_delete): the row is updated, not
removed, so a following op must not fail test_active(). The bit is only
set after a heap_rnext(), so a plain single-row UPDATE never reaches this.
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

Fixes according to Kristian Nielsen's review:
* Removed obsolete checks for slave thread
* Supporting slave with old MariaDB version.
  Events logged in cascade operation are additionally flagged with
  the long-standing NO_FOREIGN_KEY_CHECKS_F, so a replica that does
  not understand FK_CASCADE_EVENTS_F still disables foreign key checks
  and  does not re-execute the cascade

Also, thee are now binlog event flags to mark both original and derived
events. This will make it possible for the slave to choose whether to use
the derived events in applying or to execute the cascade operation

There is a new test rpl.rpl_fk_cascade_binlog_row_old_slave, for checking
compatibility with replication slave of old mariadb version
Sergei Golubchik
w 6.5 copy more tests over
Alessandro Vetere
MDEV-41036 buf_page_peek_if_young() does not properly compute and wrap page age in young calculation

buf_page_t::freed_page_clock is a 31-bit bitfield holding the low bits
of buf_pool.freed_page_clock. buf_page_peek_if_young() evaluated the
condition now < stamp + window in size_t arithmetic, which failed to
account for wraparound within the 31-bit clock space.

When the global clock wrapped past a stamp (particularly when stamp was
high, near 2^31), the comparison evaluated to true for an extended
period even though the page was older than the window. During these
wraparound windows, pages were falsely reported as young.

Because returning true causes InnoDB to skip moving pages to the MRU
head (to reduce lock contention), these falsely "young" pages were not
promoted.
As a result, active/hot pages could sink into the old portion of the LRU
list and be evicted prematurely. Additionally, buf_read_ahead_random()
over-counted recently accessed pages during wraparound intervals.

Compute age as (now - stamp) & clock_mask using 31-bit modular
arithmetic and compare age < window. A page then cleanly leaves the
young window after the intended number of evictions regardless of clock
wraparound.
Sergei Golubchik
cleanup: add const
sjaakola
MDEV-38243 Write binlog row events for changes done by cascading FK operations

This commit implements a feature which changes the handling of cascading foreign
key operations to write the changes of cascading operations into binlog.
The applying of such transaction, in the slave node, will apply just the binlog
events, and does not execute the actual foreign key cascade operation.
This will simplify the slave side replication applying and make it more predictable
in terms of potential interference with other parallel applying happning
in the node.

This feature can be turned ON/OFF by new variable:
rpl_use_binlog_events_for_fk_cascade, with default value OFF

The actual implementation is largely by windsurf.

The commit has also mtr tests for testing rpl_use_binlog_events_for_fk_cascade
feature:  rpl.rpl_fk_cascade_binlog_row, rpl.rpl_fk_set_null_binlog_row and
rpl.fk_cascade_binlog_row_rollback
Sergei Golubchik
a.test: fix encoding
Aleksey Midenkov
MDEV-41050 Aria/MyISAM: versioned DELETE via row_end index leaves a row undeleted

A system-versioned DELETE on a MyISAM or Aria table indexed by row_end
could leave the last current row undeleted. The delete scans the current
rows via an equality search on row_end = MAX, which is driven by
mi_rnext_same/maria_rnext_same. That function keeps the search's
reference key in lastkey2 and uses the HA_STATE_RNEXT_SAME flag to
remember it has already stored it. This reference-key mechanism is
exactly what lets the scan modify rows it is walking without skipping
them.

Deleting a versioned row is an in-place update of row_end, and the update
reuses lastkey2 as scratch space for the changed key, so it clears
HA_STATE_RNEXT_SAME to request that rnext_same re-store its reference on
the next call. However, TABLE::delete_row wraps the update in
HA_EXTRA_REMEMBER_POS/HA_EXTRA_RESTORE_POS, and RESTORE_POS restored the
whole saved info->update word, resurrecting the HA_STATE_RNEXT_SAME bit
that the update had just cleared.

As a result rnext_same skipped rebuilding its reference key and compared
subsequent keys against the now-overwritten lastkey2, hitting a spurious
end-of-file and terminating the scan one row early, defeating the engine's
own protection against a Halloween-style skip.

Fixed by preserving the current HA_STATE_RNEXT_SAME bit across
RESTORE_POS instead of restoring the stale saved value.