Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
ffaba11e0e69...
ffaba11e0e695c68c38f1f3c5053535df9e49fb7
Monty
Fixed wrong error handling in handler::ha_update_row()

- Error could be wrongly ignored
fdc5dc8959f7...
fdc5dc8959f7201d10c21b7d33043b3d24f1c7f6
ParadoxV5
[skip ci] MDEV-38972: preliminarily reörganize in one `.h`–`.cc` pair

This commit forcibly reörganizes the contents in the `rpl_info_file.h`
trio of header-only files to one `rpl_info_file.h` for declarations and
one `rpl_info_file.cc` for long definitions.

Due to the files’ extensive use of C++ templates, compiling at this
commit _will_ fail with “undefined reference” linker errors.
Rather, this incomplete migration commit intends to avoid muddying the
following commits’ refactors to this relocation.

The only other change in this commit is the omission of pre-C++17 stubs
used by RocksDB, as they’re obsolete since MDEV-38213 and shall be
wiped as a part of MDEV-35462.
fc6212aba097...
fc6212aba0979e79d671dd27e6b88d943a429d56
Monty
MDEV-40029 Add support for bit fields to HEAP

This adds support for BIT_FIELD in record and keys for HEAP tables.

The HEAP engine now has HA_CAN_BIT_FIELD set in table_flags()

Multiple bugs in BIT field handling was found fixed. Some in HEAP table
code, other bugs was affecting usage of BIT fields as keys.
f5b486418ba0...
f5b486418ba0c30a99365c01d62335ffbc30dac1
Monty
Removed wrong assert on thd->lex->query_tables != table_list

This was in log_event_server.cc and I got mutiple asserts on this
in valgrind builds, when malloc() returned same address for different
table list (with free() in between)
ef4e5f3f3963...
ef4e5f3f396326f59d7c8f5738687b2486357f15
rusher
[misc] Add matrix entry that builds against latest C/C 3.3 branch
edd20f4c8bf7...
edd20f4c8bf7a585f45e80c3d52ec57943c99f7d
Lawrin Novitsky
Just moved cpp submodule to latest commit
eaffd7c5c16b...
eaffd7c5c16b255d872243729bb220bce5222311
Arcadiy Ivanov
Fix CI regressions from MDEV-38975 forward-port to main

Seven code fixes, a new test, and test re-recordings for issues found
by CI on PR #5222.

**NULL dereference in `create_tmp_field()`**: `SYS_REFCURSOR` plugin
returns NULL from `make_new_field()` (cursor values cannot be
materialized). The feature added `result->flags |= FIELD_PART_OF_TMP_UNIQUE`
without a NULL check. Added `if (result)` guard.

**xmltype identity loss and recursive CTE reclength mismatch in
`Item_type_holder::create_tmp_field_ex()`**: the blob_key dispatch now
requires both: (1) `type_handler_for_tmp_table()` returns
`blob_key_type_handler()`, AND (2) `dynamic_cast<Type_handler_blob_common*>`
confirms the original type is a native blob. Condition 1 excludes xmltype
(its override returns itself). Condition 2 excludes VARCHAR types promoted
via `varstring_type_handler()` -> `too_big_for_varchar()` ->
`blob_type_handler()`. Without condition 2, wide VARCHAR in recursive CTEs
(e.g. `cast('...' as varchar(1000))`) was promoted to `Field_blob_key` in
the main UNION DISTINCT table (`part_of_unique_key=true`) but stayed as
`Field_varchar` in the incremental table (`part_of_unique_key=false`),
causing a `reclength` mismatch assertion in
`select_union_recursive::send_data()` (`main.json_equals` crash).

**Spurious `reclength > HA_MAX_REC_LENGTH` in `pick_engine()`**: the
original `choose_engine()` (both 10.11 and upstream/main) never had a
reclength check. MDEV-38975 introduced it when replacing the
`blob_fields` condition. HEAP has no internal reclength limit --
`hp_create.c` stores `uint reclength` and allocates blocks of that size;
`max_supported_record_length()` is only checked in `unireg.cc` during
user-facing CREATE TABLE. I_S tables like SLAVE_STATUS routinely have
reclength ~880KB (13 bare `Varchar()` columns). The check forced them to
Aria where `fill_slave_status()` returned 0 rows. Removed the check and
the unused `reclength` parameter from `pick_engine()`.

**Multi-update `tmp_memory_table_size` override**: the 10.11 feature
overrode `big_tables=FALSE` for multi-update dedup tables. The forward-port
translated this as `tmp_memory_table_size=SIZE_T_MAX` when the variable
was 0. But `big_tables=FALSE` was a soft "don't force disk" hint, while
`tmp_memory_table_size=SIZE_T_MAX` overrides the user's explicit
`tmp_memory_table_size=0` directive. Since main removed `big_tables`
entirely (MDEV-19713), the override is not needed. Removed.

**Zero-length key rejection in `check_tmp_key()`**: reject `key_len == 0`
to prevent useless zero-length keys from being created by `add_tmp_key()`.
Reachable when all key parts are CHAR(0) NOT NULL: `key_length()` returns
0, the field is not nullable (no HA_KEY_NULL_LENGTH) and not
VARCHAR/BLOB/GEOMETRY (no HA_KEY_BLOB_LENGTH), so `fld_store_len` is 0
for every part. Without this guard, `check_tmp_key()` would accept the
key (0 <= max_key_length), and the optimizer would create a ref key that
cannot distinguish any rows. Added `heap.char0_key` test exercising this
via a materialized derived table with CHAR(0) NOT NULL join columns.

**Non-deterministic `column_compression` test**: HEAP blob support allows
compressed VARCHAR/TEXT temp tables to stay in HEAP instead of falling to
Aria, changing row iteration order. Added `--sorted_result` to the two
MDEV-24726 subqueries that lack `ORDER BY`.

Test changes:
- `spatial_utility_function_collect`: added ORDER BY to window function
  that lacked it (results were engine-row-order-dependent)
- `tmp_space_usage`: removed multi-update override; forced disk for
  MDEV-34016/34060 Aria-specific test sections (blob I_S tables now
  stay in MEMORY)
- `blob_update_overflow`: replaced `SHOW STATUS LIKE 'Created_tmp_%'`
  with targeted I_S query (Created_tmp_files varies on sanitizer builds)
- `column_compression`: added `--sorted_result` for MDEV-24726 queries
- `char0_key` (new): CHAR(0) NOT NULL derived table ref key rejection
- Re-recorded 8 tests for expected "temp table stays in MEMORY" changes
d98f482f2167...
d98f482f21678e9f704f9c12acbad37be1febc0a
Lawrin Novitsky
ODBC-493 if LOCAL INFILE is not disabled, it should be LOCAL_INFILE_MODE_AUTO

This is safer mode than allowing it always
d2ed3cc58b20...
d2ed3cc58b200cc8cbcb08ca54583da440ee96a2
rusher
[misc] use DOCKER_LOGIN as user if secret available
c0200a5dde9a...
c0200a5dde9a4b83d27ea21befcf51bf8955bd25
Lawrin Novitsky
Merge branch 'cpp-1.0' into cpp-1.1
b4905a78fcec...
b4905a78fcec6dc4575c8c2fd70624efded541e0
Lawrin Novitsky
Moved underlying libmariadb to 3.4 series - v3.4.9 atm

We have to support parsec auth plugin as minimum. zero config TLS is
also a big plus, but we don't change related options defaults - the
certificate is not yet validated by default.
a7a9df248d22...
a7a9df248d224a418cab01e45e15f63918f1e3a1
Lawrin Novitsky
Merge branch 'cpp-1.0' into cpp-1.1
a7306b8130e8...
a7306b8130e8ba2c1de2b6d7873d44c0a3bc0c45
ParadoxV5
MDEV-38972: Reorganize `Info_file` entities

This commit consolidate the value structs and their related functions
to shrink the class hierarchy and mitigate many templates’s limitations.

* Rename `Persistence` to `Value_interface` and add template `Value`
* Replace subclasses with explicit specializations of the
  `Value` template’s instance methods wherever available
  * This ensures `Value` is used literally as a template – `#define`
    but typed – rather than merely for the generics as much as possible.
* Move `mariadbd_option` reference template parameter to
  `Value::default_value` pointer instance variable
* Merge DEFAULT-less classes to a case of the optional `Value`
template where the `Value::default_value` is `nullptr`
* Replace `trilean` with `std::optional<bool>` from the template
* Rename `enum_master_use_gtid::DEFAULT` to `AUTO` and *intend*
  `std::nullopt` to represent the unset/DEFAULT state for
  `Master_info_file::master_use_gtid` instead
* Dissolve `Int_IO_CACHE` to one `my_b_getsll()` (at least for now)
  and the rest merged into numeric `Value` instance methods
* Split `Master_info_file::Heartbeat_period_value`’s static methods
  to instance methods of a new `Uint32_3` conversion wrapper

Since the publication of MDEV-37530, my unimplemented designs have
evolved to feature split “metaclasses” that it can share with
`information_schema.slave_status`, especially after MDEV-31342.
As such, this commit’s redesign is modelled with a
neutral `Value::default_value` to reflect how NOT NULL
is a column constraint rather than a value property.

While here, the commit also fixes some nits in the code documentation.
912330a34801...
912330a3480167c988803205e6897870e639a91f
Daniel Bartholomew
bump the VERSION
8cd8e945a67c...
8cd8e945a67c362a6c2f0b044d46aca6e516eb2c
Lawrin Novitsky
Introduced UBSAN.supp to supress UBSAN issues beyond our control

(on C/C side). They are quite the same as in C/ODBC
87b35f520ca7...
87b35f520ca7b01b61db3ebd064be4ae1db9b495
Monty
fixup! remove is_text_key_segment
7f78019099e5...
7f78019099e5cade252781b4d5f3fb666905cb7b
Arcadiy Ivanov
MDEV-38975: HEAP engine BLOB/TEXT/JSON/GEOMETRY support with indexable blob columns

Remove the HA_NO_BLOBS restriction from the HEAP engine, allowing
the optimizer to keep temporary tables with BLOB/TEXT columns in
memory when they fit within max_heap_table_size / tmp_memory_table_size
limits.  Additionally, advertise HA_CAN_GEOMETRY so explicit
CREATE TABLE ... ENGINE=MEMORY with GEOMETRY columns works.

Unlike other HEAP blob implementations (e.g. Percona), this patch
provides full HASH index support on blob columns, enabling efficient
lookups, GROUP BY, and DISTINCT operations directly in HEAP without
falling back to disk.

Architecture
------------

BLOB data is stored using continuation records -- additional fixed-size
records allocated from the same HP_BLOCK that holds regular rows.  This
reuses existing allocation, free list, and size accounting with minimal
structural change, and avoids per-blob my_malloc() calls.

The existing single-byte visibility flag is extended into a flags byte
with bits for HP_ROW_HAS_CONT, HP_ROW_IS_CONT, HP_ROW_CONT_ZEROCOPY,
HP_ROW_SINGLE_REC, and HP_ROW_MULTIPLE_REC.  Continuation records are
grouped into variable-length runs -- contiguous sequences within a leaf
block.  Only the first record of each run carries a 10-byte header
(next_cont pointer + run_rec_count); inner records are pure payload.

Three storage formats, detected by flag bits via inline predicates:

  Case A (HP_ROW_SINGLE_REC): single record, no header, data at
  offset 0.  Zero-copy read.

  Case B (HP_ROW_CONT_ZEROCOPY): single run, multiple records.
  Header in rec 0, data contiguous in rec 1..N-1.  Zero-copy read
  via chain + recbuffer.

  Case C (HP_ROW_MULTIPLE_REC): one or more runs linked via
  next_cont.  Reassembly into blob_buff required.

Run allocation uses a two-phase strategy: (1) peek-then-unlink walk
of the free list detecting contiguous groups, (2) tail allocation
from HP_BLOCK for remaining data.  A Step 3 scavenge fallback
walks the entire free list when tail allocation fails.

HP_SHARE::total_records tracks all physical records (primary +
continuation), while HP_SHARE::records remains the logical count
used by hash bucket mapping.

Reassembly buffer (HP_INFO::blob_buff) follows the same pattern as
InnoDB's blob_heap -- allocated once, grown via my_realloc, freed
on heap_reset()/close.  Zero-copy cases (A/B) return pointers
directly into HP_BLOCK with no copy.

Full HASH index key handling for BLOB columns: hp_rec_hashnr(),
hp_rec_key_cmp(), hp_key_cmp(), hp_make_key(), hp_hashnr() are
extended for HA_BLOB_PART segments.  Hash pre-check optimization
skips expensive blob materialization when hashes differ.  PAD SPACE
collation semantics are preserved for blob key comparisons.

Field_blob_key (Monty) produces HEAP-native key format (4-byte length
+ 8-byte data pointer) directly, eliminating key buffer translation
between the SQL layer and HEAP engine.

SQL layer changes
-----------------

pick_engine() (new, extracted from choose_engine()): replaced the
blob_fields check with a reclength > HA_MAX_REC_LENGTH guard.
choose_engine() calls pick_engine() with the real reclength.
pick_engine() is also called early in finalize() with reclength=0
to predict whether the engine will be HEAP, enabling blob-aware
GROUP BY key setup that avoids unnecessary m_using_unique_constraint.

finalize(): HEAP+blob uses fixed-width rows; GROUP BY key setup sets
key_part_flag from field, uses item max_length for blob key sizing.
store_length initialized for all GROUP BY key parts.  key_type uses
field->binary() to determine FIELDFLAG_BINARY vs text collation.
DISTINCT key setup skips null-bits helper for HEAP.

remove_duplicates(): blob check moved before HEAP check to fall
through to remove_dup_with_compare().

Aggregator_distinct::add(): overflow-to-disk conversion via
create_internal_tmp_table_from_heap() for non-dup write errors.

Expression cache disabled for HEAP+blob (key format incompatibility).

FULLTEXT early detection in mysql_derived_prepare(): forces disk
engine via TMP_TABLE_FORCE_MYISAM when outer query uses MATCH.

Deferred blob chain free (MDEV-39732): heap_delete() saves chain
pointers to pending_blob_chains, flushed on next mutation or
heap_reset()/close.  Prevents dangling zero-copy pointers during
binlog_log_row().

REPLACE safety (MDEV-39825): HP_SHARE::write_can_replace flag
forces copy mode in hp_read_blobs(), preventing blob data corruption
from freed-then-reused continuation records during REPLACE.

Geometry GROUP_CONCAT fix (MDEV-39761): downgrade Field_geom to
Field_blob for GROUP_CONCAT temp tables in both expression creation
paths.  Type_handler_geometry::type_handler_for_tmp_table() added.

Geometry GROUP BY key fix (MDEV-39871): detect when new_key_field()
produced non-blob Field_varstring for a blob column, replace with
Field_blob_key.

Performance
-----------

Non-blob tables: zero regression.  Every blob-specific code path is
guarded by if (share->blob_count).  No new allocations, no format
changes, no hash function changes for non-blob keys.

Blob tables: eliminates file creation/deletion overhead and page cache
management.  For single-run blobs (common case), the read path is
entirely zero-copy.

Limitations
-----------

1. No BTREE indexes on blob columns (HASH only)
2. No partial-key prefix indexing for blobs
3. 2x memory for Case C reads only (A/B are 1x)
4. No blob compression
5. 65,535 records per run (uint16 cap, auto-split)
6. max_heap_table_size applies to continuation records
7. Expression cache disabled for HEAP+blob
8. FULLTEXT forces disk engine

Linked bugs fixed:

- MDEV-39703: mroonga fulltext test ordering
- MDEV-39723: ER_DUP_ENTRY on GROUP BY with blob column
- MDEV-39724: crash in hp_is_single_rec with GROUP BY
- MDEV-39732: slave crash in hp_free_run_chain on blob replication
- MDEV-39761: Field_geom::store() assertion in GROUP_CONCAT
- MDEV-39782: RBR ER_KEY_NOT_FOUND on HEAP blob UPDATE
- MDEV-39825: blob data corruption on REPLACE into HEAP table
- MDEV-39871: crash in my_hash_sort_bin on GROUP BY with geometry

Reviewed by: Michael Widenius <[email protected]>
  Monty reviewed the entire patch. Areas where he suggested changes
  or contributed code:
  - Field_blob_key class (HEAP-native blob key format, 4-byte length +
    data pointer)
  - Duplicate key fix on HEAP-to-Aria conversion
  - hp_blob_key_length() uint32 fix
  - hp_rec_hashnr_stored removal
  - type_handler_for_tmp_table() param cleanup
  - Type_handler_geometry::type_handler_for_tmp_table() virtual
  - blob pointer bzero()
  - find_unique_row() double-materialization fix
  - Tail reclaim review
  - Batch tail allocation review
  - hp_update.c cleanup
  - Field_blob_compressed temp table fix
  - row_pack_length() dedup
  - pack_length_no_ptr() removal
  - Race condition fix in HEAP
  - MDEV-39703 mroonga test fix
  - MDEV-39825 write_can_replace optimization
  - is_text_key_segment removal (field->binary() simplification)
  - Documentation (Docs/internal-temporary-tables.txt)
Contribution by: Alexander Barkov <[email protected]>
  Type_handler::make_and_init_table_field_ex() -- refactored temp table
  field creation from inline code in sql_select.cc into type handler
  virtual methods (sql_type.cc, sql_type_geom.cc), enabling clean
  per-type-handler field creation for HEAP blob promotion.
72a6c5abbf21...
72a6c5abbf214b2c57a39437a14d0383d5a1d489
Lawrin Novitsky
Fix of UBSAN errors.

In one place copying from nullptr was possible. It could happen if the
streamed text resultset had to be cached(if the connection is requested
for other operation), and NULL value was present.
In couple of places int64_t could be overflown - had to be changed to
uint64 as it is defined plus othe implied changes.
In particular fetching MIN_INT64 value in text protocol could cause
that. Looks like all compilers work fine in this case as there is the
test for this.
71d7543651dd...
71d7543651dd058a0054d520dad90c5e7e041c42
Lawrin Novitsky
One more UBSAN silencing.

Same nature "function called using wrong pointer type" for C/C parameters
callback call. In this case besides inclusion in the namespace the
function had accidentally wrong return type.
70a62a7fcab3...
70a62a7fcab38c68bb41249598dbb2d52eaffd6a
Arcadiy Ivanov
Fix CI regressions from MDEV-38975 forward-port to main

Six code fixes and test re-recordings for issues found by CI on PR #5222.

**NULL dereference in `create_tmp_field()`**: `SYS_REFCURSOR` plugin
returns NULL from `make_new_field()` (cursor values cannot be
materialized). The feature added `result->flags |= FIELD_PART_OF_TMP_UNIQUE`
without a NULL check. Added `if (result)` guard.

**xmltype identity loss and recursive CTE reclength mismatch in
`Item_type_holder::create_tmp_field_ex()`**: the blob_key dispatch now
requires both: (1) `type_handler_for_tmp_table()` returns
`blob_key_type_handler()`, AND (2) `dynamic_cast<Type_handler_blob_common*>`
confirms the original type is a native blob. Condition 1 excludes xmltype
(its override returns itself). Condition 2 excludes VARCHAR types promoted
via `varstring_type_handler()` -> `too_big_for_varchar()` ->
`blob_type_handler()`. Without condition 2, wide VARCHAR in recursive CTEs
(e.g. `cast('...' as varchar(1000))`) was promoted to `Field_blob_key` in
the main UNION DISTINCT table (`part_of_unique_key=true`) but stayed as
`Field_varchar` in the incremental table (`part_of_unique_key=false`),
causing a `reclength` mismatch assertion in
`select_union_recursive::send_data()` (`main.json_equals` crash).

**Spurious `reclength > HA_MAX_REC_LENGTH` in `pick_engine()`**: the
original `choose_engine()` (both 10.11 and upstream/main) never had a
reclength check. MDEV-38975 introduced it when replacing the
`blob_fields` condition. HEAP has no internal reclength limit --
`hp_create.c` stores `uint reclength` and allocates blocks of that size;
`max_supported_record_length()` is only checked in `unireg.cc` during
user-facing CREATE TABLE. I_S tables like SLAVE_STATUS routinely have
reclength ~880KB (13 bare `Varchar()` columns). The check forced them to
Aria where `fill_slave_status()` returned 0 rows. Removed the check and
the unused `reclength` parameter from `pick_engine()`.

**Multi-update `tmp_memory_table_size` override**: the 10.11 feature
overrode `big_tables=FALSE` for multi-update dedup tables. The forward-port
translated this as `tmp_memory_table_size=SIZE_T_MAX` when the variable
was 0. But `big_tables=FALSE` was a soft "don't force disk" hint, while
`tmp_memory_table_size=SIZE_T_MAX` overrides the user's explicit
`tmp_memory_table_size=0` directive. Since main removed `big_tables`
entirely (MDEV-19713), the override is not needed. Removed.

**Zero-length key rejection in `check_tmp_key()`**: defense-in-depth
guard rejecting `key_len == 0` to prevent useless zero-length keys from
being created by `add_tmp_key()`.

**Non-deterministic `column_compression` test**: HEAP blob support allows
compressed VARCHAR/TEXT temp tables to stay in HEAP instead of falling to
Aria, changing row iteration order. Added `--sorted_result` to the two
MDEV-24726 subqueries that lack `ORDER BY`.

Test changes:
- `spatial_utility_function_collect`: added ORDER BY to window function
  that lacked it (results were engine-row-order-dependent)
- `tmp_space_usage`: removed multi-update override; forced disk for
  MDEV-34016/34060 Aria-specific test sections (blob I_S tables now
  stay in MEMORY)
- `blob_update_overflow`: replaced `SHOW STATUS LIKE 'Created_tmp_%'`
  with targeted I_S query (Created_tmp_files varies on sanitizer builds)
- Re-recorded 8 tests for expected "temp table stays in MEMORY" changes
- `column_compression`: added `--sorted_result` for MDEV-24726 queries
6f9da32b9f60...
6f9da32b9f6052e16f3ec4d0e85bc9dd6f6ecc08
Sergei Petrunia
Code cleanup (3).
638f160a5b96...
638f160a5b96b2336ca26204b55bd60983239590
Monty
Removed wrong assert on thd->lex->query_tables != table_list

This was in log_event_server.cc and I got mutiple asserts on this
in valgrind builds, when malloc() returned same address for different
table list (with free() in between)
603ea69bf584...
603ea69bf584450716f5dbf3bf3d2cc2b7597e3e
Lawrin Novitsky
Removed github actions config as this branch is not intended for testing

there.
There are other things that don't need to be here, but keeping them not
to make merges even more cumbersome.
560d9b2c65ac...
560d9b2c65ac5f2c9980f7df3b8620d547ad7d45
Monty
Added support for bit fields and CHECK TABLE to the memory engine

HEAP engine now has HA_CAN_BIT_FIELD set in table_flags()
CHECK TABLE is supported, but will only return ok or failed.
Still good enough for testing heap table consistenty.

Note that one bug for bit field in HEAP tables was fixed as part of
MDEV-38975

Bugs fixed:
Field_bit::get_key_image() returned wrong length for lengths
8,16,32,48,64.
hp_rb_make_key() did not handle bit fields correctly.

Other things:
- Added sorted_order to some test in type_bit to be able to run it
  with the memory engine
53756b021e22...
53756b021e2260a686be807a39c5e2787fbee9cb
Lawrin Novitsky
Merge branch 'cpp-1.1' into mini
521136a25cff...
521136a25cff90376a1cd42a2d31a76bf25fae3e
Lawrin Novitsky
ODBC-494 Fix of string escaping similar to CONCPP-153

Escaping did not consider if the string is in multibyte charsets. That
could affect client side prepared statements with binary parameters.

Also, there was another problem with binary parameters in this case -
the string literals were not marked as _binary

The test has been added.
4b2deba35b13...
4b2deba35b13329a3838370e4bc1ad683f13b190
Razvan Liviu Varzaru
Support fedora for static_test linking
48037682e027...
48037682e02797148ce8e321bc80cec6d0c38445
rusher
[misc] add maxscale testing to CI
46488f191396...
46488f191396c1396e593e62250de49700f5ddef
Lawrin Novitsky
Merge branch 'cpp-1.0' into cpp-1.1
42480479e189...
42480479e1898d1388eb732d60719f3bf4d9bac6
Lawrin Novitsky
Merge branch 'odbc-3.1'
4176e0324bf5...
4176e0324bf563b436509bee203c72b74476bb8e
Giorgio Caculli
required <cstdint> inclusion in various headers
3948339afb7f...
3948339afb7fb0ebc28a6a59021383b495ffc293
Lawrin Novitsky
CONCPP-153 Parameter value escaping did not consider the charset

In case of client side statement prepare, if the data is encoded in multibyte
charsets, escaping of string and binary parameter could lead to incorrect data
that in its turn could let sql injection with charsets like big5, gbk, sjis, cp932.

The fix considers the charset now. The testcase has been added.
38c74af6ab05...
38c74af6ab054ad7d17595997aa9d7d1855a3ebd
rusher
[misc] use DOCKER_LOGIN as user if secret available
2d871f9c7fd1...
2d871f9c7fd1df1b20860297083ab1539a24f3d4
Arcadiy Ivanov
MDEV-38975: HEAP engine BLOB/TEXT/JSON/GEOMETRY support with indexable blob columns

Remove the HA_NO_BLOBS restriction from the HEAP engine, allowing
the optimizer to keep temporary tables with BLOB/TEXT columns in
memory when they fit within max_heap_table_size / tmp_memory_table_size
limits.  Additionally, advertise HA_CAN_GEOMETRY so explicit
CREATE TABLE ... ENGINE=MEMORY with GEOMETRY columns works.

Unlike other HEAP blob implementations (e.g. Percona), this patch
provides full HASH index support on blob columns, enabling efficient
lookups, GROUP BY, and DISTINCT operations directly in HEAP without
falling back to disk.

Architecture
------------

BLOB data is stored using continuation records -- additional fixed-size
records allocated from the same HP_BLOCK that holds regular rows.  This
reuses existing allocation, free list, and size accounting with minimal
structural change, and avoids per-blob my_malloc() calls.

The existing single-byte visibility flag is extended into a flags byte
with bits for HP_ROW_HAS_CONT, HP_ROW_IS_CONT, HP_ROW_CONT_ZEROCOPY,
HP_ROW_SINGLE_REC, and HP_ROW_MULTIPLE_REC.  Continuation records are
grouped into variable-length runs -- contiguous sequences within a leaf
block.  Only the first record of each run carries a 10-byte header
(next_cont pointer + run_rec_count); inner records are pure payload.

Three storage formats, detected by flag bits via inline predicates:

  Case A (HP_ROW_SINGLE_REC): single record, no header, data at
  offset 0.  Zero-copy read.

  Case B (HP_ROW_CONT_ZEROCOPY): single run, multiple records.
  Header in rec 0, data contiguous in rec 1..N-1.  Zero-copy read
  via chain + recbuffer.

  Case C (HP_ROW_MULTIPLE_REC): one or more runs linked via
  next_cont.  Reassembly into blob_buff required.

Run allocation uses a two-phase strategy: (1) peek-then-unlink walk
of the free list detecting contiguous groups, (2) tail allocation
from HP_BLOCK for remaining data.  A Step 3 scavenge fallback
walks the entire free list when tail allocation fails.

HP_SHARE::total_records tracks all physical records (primary +
continuation), while HP_SHARE::records remains the logical count
used by hash bucket mapping.

Reassembly buffer (HP_INFO::blob_buff) follows the same pattern as
InnoDB's blob_heap -- allocated once, grown via my_realloc, freed
on heap_reset()/close.  Zero-copy cases (A/B) return pointers
directly into HP_BLOCK with no copy.

Full HASH index key handling for BLOB columns: hp_rec_hashnr(),
hp_rec_key_cmp(), hp_key_cmp(), hp_make_key(), hp_hashnr() are
extended for HA_BLOB_PART segments.  Hash pre-check optimization
skips expensive blob materialization when hashes differ.  PAD SPACE
collation semantics are preserved for blob key comparisons.

Field_blob_key (Monty) produces HEAP-native key format (4-byte length
+ 8-byte data pointer) directly, eliminating key buffer translation
between the SQL layer and HEAP engine.

SQL layer changes
-----------------

choose_engine(): removed blob_fields check, added reclength >
HA_MAX_REC_LENGTH.

finalize(): HEAP+blob uses fixed-width rows; GROUP BY key setup sets
key_part_flag from field, uses item max_length for blob key sizing.
store_length initialized for all GROUP BY key parts.  DISTINCT key
setup skips null-bits helper for HEAP.

remove_duplicates(): blob check moved before HEAP check to fall
through to remove_dup_with_compare().

Aggregator_distinct::add(): overflow-to-disk conversion via
create_internal_tmp_table_from_heap() for non-dup write errors.

Expression cache disabled for HEAP+blob (key format incompatibility).

FULLTEXT early detection in mysql_derived_prepare(): forces disk
engine via TMP_TABLE_FORCE_MYISAM when outer query uses MATCH.

Deferred blob chain free (MDEV-39732): heap_delete() saves chain
pointers to pending_blob_chains, flushed on next mutation or
heap_reset()/close.  Prevents dangling zero-copy pointers during
binlog_log_row().

REPLACE safety (MDEV-39825): HP_SHARE::write_can_replace flag
forces copy mode in hp_read_blobs(), preventing blob data corruption
from freed-then-reused continuation records during REPLACE.

Geometry GROUP_CONCAT fix (MDEV-39761): downgrade Field_geom to
Field_blob for GROUP_CONCAT temp tables in both expression creation
paths.  Type_handler_geometry::type_handler_for_tmp_table() added.

Geometry GROUP BY key fix (MDEV-39871): detect when new_key_field()
produced non-blob Field_varstring for a blob column, replace with
Field_blob_key.

Performance
-----------

Non-blob tables: zero regression.  Every blob-specific code path is
guarded by if (share->blob_count).  No new allocations, no format
changes, no hash function changes for non-blob keys.

Blob tables: eliminates file creation/deletion overhead and page cache
management.  For single-run blobs (common case), the read path is
entirely zero-copy.

Limitations
-----------

1. No BTREE indexes on blob columns (HASH only)
2. No partial-key prefix indexing for blobs
3. 2x memory for Case C reads only (A/B are 1x)
4. No blob compression
5. 65,535 records per run (uint16 cap, auto-split)
6. max_heap_table_size applies to continuation records
7. Expression cache disabled for HEAP+blob
8. FULLTEXT forces disk engine

Linked bugs fixed:

- MDEV-39703: mroonga fulltext test ordering
- MDEV-39723: ER_DUP_ENTRY on GROUP BY with blob column
- MDEV-39724: crash in hp_is_single_rec with GROUP BY
- MDEV-39732: slave crash in hp_free_run_chain on blob replication
- MDEV-39761: Field_geom::store() assertion in GROUP_CONCAT
- MDEV-39782: RBR ER_KEY_NOT_FOUND on HEAP blob UPDATE
- MDEV-39825: blob data corruption on REPLACE into HEAP table
- MDEV-39871: crash in my_hash_sort_bin on GROUP BY with geometry

Reviewed by: Michael Widenius <[email protected]>
  Monty reviewed the entire patch. Areas where he suggested changes
  or contributed code:
  - Field_blob_key class (HEAP-native blob key format, 4-byte length +
    data pointer)
  - Duplicate key fix on HEAP-to-Aria conversion
  - hp_blob_key_length() uint32 fix
  - hp_rec_hashnr_stored removal
  - type_handler_for_tmp_table() param cleanup
  - Type_handler_geometry::type_handler_for_tmp_table() virtual
  - blob pointer bzero()
  - find_unique_row() double-materialization fix
  - Tail reclaim review
  - Batch tail allocation review
  - hp_update.c cleanup
  - Field_blob_compressed temp table fix
  - row_pack_length() dedup
  - pack_length_no_ptr() removal
  - Race condition fix in HEAP
  - MDEV-39703 mroonga test fix
  - MDEV-39825 write_can_replace optimization
  - Documentation (Docs/internal-temporary-tables.txt)
Contribution by: Alexander Barkov <[email protected]>
  Type_handler::make_and_init_table_field_ex() -- refactored temp table
  field creation from inline code in sql_select.cc into type handler
  virtual methods (sql_type.cc, sql_type_geom.cc), enabling clean
  per-type-handler field creation for HEAP blob promotion.
225fa820fb4f...
225fa820fb4f29c38a8fd0db5dd47517f68352e8
Dylan Liu
fix: correct CMake path configuration error that caused build failures when used as a subdirectory.
2022c5548ad4...
2022c5548ad44741d842ed9b3862c1ee7d70e917
Lawrin Novitsky
Increased cmake min required version to enable build with newest cmake's

Also, fixed UBSAN issue caused by including mysql header inside the
namespace and using pointer to destroyer functions in the smart
pointers. Tha was in ConnectProtocol and TextRowProtocol classes with
MYSQL and MYSQL_RES handles, respectively. Removed unique_ptr use
completely as it's pretty safe and easy to destroy them in the the
respective class destructor.
Added check and exception throw if MYSQL handle could not be allocated -
taht was missing.
15d71ede6798...
15d71ede6798a8e016943e723c5e7c600109c4be
Monty
fixup! remove is_text_key_segment
147a2f9256ff...
147a2f9256ff9f46b4113c36567a7d16d1111169
Monty
MDEV-40030 Add support for CHECK TABLE for memory tables

CHECK TABLE is now supported, but will only return ok or fail.
Still good enough for testing heap table consistenty.
01701aa3c3d6...
01701aa3c3d631d5ec15e508267396bb2603a98a
Lawrin Novitsky
More fixes to please UBSAN and MSAN.

For UBSAN it's the similar problem as in prvious commit, for MSAN - it
suddenly had issues afte move of min required cmake version to 3.5.
Seemingly it is connected to policy 0056 and it's new behavior.
013ca138c318...
013ca138c318a95aa57fba077b5d43ed650d4d22
Sergei Petrunia
More code cleanups.
BuildBot (0.8.9) working for the MariaDB project.
Page built: Sun 14 Jun 2026 02:28:40 (UTC)