Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Monty
Ensure that 'param' is always set for type_handler_for_tmp_table()
Monty
Set blob pointer to null with bzero()

This is to avoid storing a pointer into an unaligned address.
Arcadiy Ivanov
MDEV-39782: fix row-based replication of UPDATE on HEAP blob tables

`handler::ha_update_row()` calls `binlog_log_row()` AFTER
`update_row()` returns. For HEAP blob tables, `heap_update()` was
freeing old continuation chains immediately via `hp_free_run_chain()`,
overwriting chain records with `del_link` pointers. The zero-copy
pointers in the before-image record (used by the binlog to read blob
data) became dangling, corrupting the replication event. The slave
then failed to locate the row with `ER_KEY_NOT_FOUND`.

Two corruption vectors:

1. **Dangling zero-copy pointers**: `hp_free_run_chain()` zeroes chain
  records that `record[1]` (before-image) still references via
  zero-copy (Case A/B) pointers.

2. **`blob_buff` reallocation**: `hp_read_blobs()` called to refresh
  `record[0]` (after-image) could `realloc` `info->blob_buff`,
  invalidating Case C materialized pointers in `record[1]`.

**Fix**:

- Split the chain-free path by `share->internal`: internal temporary
  tables (never binlogged) free chains immediately; user-created tables
  defer via `pending_blob_chains`, matching the DELETE path from
  MDEV-39732.

- Gate `hp_read_blobs()` refresh on `share->internal`: user tables
  skip the refresh since the SQL layer re-materializes on the next
  fetch, avoiding the `blob_buff` reallocation hazard.

- Move `hp_flush_pending_blob_free()` in `heap_write()` from entry to
  after `hp_write_blobs()` completes, so that blob data in the record
  buffer remains readable during the write. This prevents use-after-free
  when `UPDATE ... FOR PORTION OF` inserts a row whose blob pointers
  reference deferred chains.
Monty
Review of "Replace `hp_blob_run_format()` enum with direct bit testing"

Change 'uchar format' to 'uint8 format' to make it clear that we use
format as a number.
Monty
Cleanup of previous commit
- Added virtual function Type_handler_geometry::type_handler_for_tmp_table()
  to be able to remove testing of MYSQL_TYPE_GEOMETRY in
  Item::tmp_table_field_from_field_type()

Co-author: Alexander Barkov <[email protected]>
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

In progress
Monty
fixup! 13e1fdfb6d3deb492f81e07caacadc2c4fa75dfb

Fixed duplicate key error when converting HEAP table Aria
Monty
Review fixes for "Batch tail allocation for blob continuation chains"

- Optimized setting is_only_run (avoiding one/two if's for each loop)
- Removed test for run_rec_count == 1 in a context where it was guaranteed
  to be 1.
- Removed not needed variable prev_pos (as it was always identical to
  run_start). This also fixes one of the bugs fixed by the next commit.
Monty
Review commit of Avoid double blob materialization
Arcadiy Ivanov
Replace `hp_blob_run_format()` enum with direct bit testing

Remove `enum hp_blob_format` and `hp_blob_run_format()` indirection.
Add `HP_ROW_MULTIPLE_REC` (bit 5) so all three blob storage formats
have a dedicated flag bit. Add named inline predicates
`hp_is_single_rec()`, `hp_is_zerocopy()`, `hp_is_multi_run()` matching
the existing `hp_is_active()`/`hp_has_cont()`/`hp_is_cont()` pattern.

Change `hp_write_run_data()` format parameter from enum to `uchar`
receiving bit constants directly; simplify flags byte assignment from
ternary to bitwise OR.

Addresses review feedback F127-F128, F130, F132-F134.
Arcadiy Ivanov
Replace internal Case A/B/C terminology with descriptive names in blob_find_unique test
Monty
Small performance update of previous commit
Rex Johnston
MDEV-39495 Parallel Query: Use temporary work tables to ship results: basic test

Built on top of MDEV-39492, the parallel worker manager creates N
temporary tables and gives them to each parallel worker to populate.
At the conclusion of all worker threads, each row from the above tables
is added to the join_tab->table representing the query result.  To use

MariaDB [test]> set session parallel_worker_threads=10;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> select SQL_BUFFER_RESULT * from t1;
+-------+------+
| a    | b    |
+-------+------+
| 12345 | test |
| 12345 | test |
| 12345 | test |
| 12345 | test |
| 12345 | test |
| 12345 | test |
| 12345 | test |
| 12345 | test |
| 12345 | test |
| 12345 | test |
|    1 | one  |
|    2 | two  |
|    1 | un  |
+-------+------+
13 rows in set (1.989 sec)

rows where a=12345 are added by the parallel workers, otherwise
they are from t1;
Monty
Review of reclaim tail records on failed blob allocation

- Simplify loop in hp_shrink_tail to avoid multiplcations
- Moved assert from callers of hp_shrink_tail() to hp_shrink_tail()
Arcadiy Ivanov
MDEV-39761: fix `Field_geom::store()` assertion in GROUP_CONCAT with geometry expressions

`Field_geom::store()` asserts `!table->blob_storage`, but GROUP_CONCAT
with ORDER BY or DISTINCT sets `blob_storage` on its internal temp table.
When geometry expressions (IF, CASE, COALESCE, DEFAULT, etc.) appear as
GROUP_CONCAT arguments, the type handler field creation path creates
`Field_geom` directly, bypassing the existing downgrade in
`Field_blob::make_new_field()` that converts `Field_geom` to plain
`Field_blob` for GROUP_CONCAT temp tables.

Fix: downgrade geometry to `Field_blob` (via `type_handler_long_blob`) in
both field creation paths that expressions can take:
- `create_tmp_field_ex_from_handler()` for `Item_result_field` items
- `tmp_table_field_from_field_type()` for `Item_default_value` and other
  callers that bypass the handler path
Arcadiy Ivanov
MDEV-39723, MDEV-39724: fix `hp_key_cmp()` blob packlength

`hp_key_cmp()` hardcoded `packlength=4` for the record side of blob
segment comparison. This is wrong when `Field_blob_key` preserves
the original blob subtype's packlength (1-4) via
`set_pack_length(blob_handler->length_bytes())`.

Use `seg->bit_start` (the actual packlength) instead. Update comments
in `hp_key_cmp()` and `hp_make_key()` to document the asymmetry
between record-side format (packlength-byte length prefix) and
key-side format (always 4-byte normalized length from `hp_make_key()`).
Monty
Review commit for Overflow-to-Aria on `ha_update_tmp_row()`...

- Updated test to print number created temporary files for all
  test parts.
- Fixed long comment in hp_blob.c
Sergei Petrunia
MDEV-39768: Optimizer trace should use utf8mb4

JSON is utf8mb4. Change INFORMATION_SCHEMA.OPTIMIZER_TRACE.TRACE from
using utf8mb3 to mb4.

In order to do that:
- Add m_charset into Show::Type (NULL means "use the default")
- Make Type_handler_varchar::make_schema_field() check that field
  and use the specified charset/collation.
- Add a constructor to Show::Longtext that allows to specify the charset
- Make optimizer_trace use my_charset_utf8mb4_general_ci.
- In fill_optimizer_trace_info(), indicate that the trace we've
  collected is utf8mb4
Arcadiy Ivanov
fixup! c7643a97878d170f43856d4839500f59e4d6cc2f

Fix typo: "where" -> "were" in hp_shrink_tail() comment
Arcadiy Ivanov
MDEV-39732: fix row-based replication of DELETE on HEAP blob tables

`handler::ha_delete_row()` calls `binlog_log_row()` AFTER
`delete_row()` returns. For HEAP blob tables, `heap_delete()` was
freeing continuation chains immediately, overwriting chain records
with free-list `del_link` pointers. The zero-copy pointers in the
record buffer (used by the binlog to read blob data) became dangling,
causing corrupted replication events.

**Fix**: split the blob free path by table type:

- **Internal temporary tables** (`share->internal`): free chains
  immediately via `hp_free_blobs()` — these are never binlogged.

- **User-created MEMORY tables**: defer the free by saving chain
  head pointers into `HP_INFO::pending_blob_chains` (one per blob
  column, allocated once at `heap_open_from_share()`). The flush
  runs on the next mutating operation (`heap_write`, `heap_update`,
  `heap_delete`) or on `heap_reset`/`heap_close`.

`heap_clear()` (TRUNCATE) invalidates `has_pending_blob_free`
without flushing, since the HP_BLOCK tree is already destroyed.

Also fixes `hp_free_blobs()` to skip zero-length blobs (avoids
chasing a NULL chain pointer) and `heap_update()` to NULL the
chain pointer slot for unchanged zero-length blobs (prevents
stale SQL-layer pointers from being misread as chain heads).
Sergei Petrunia
MDEV-39720: Optimizer Trace doesn't quote string values, may have invalid JSON

- Make Json_writer::add_str() do the quoting with json_escape_to_string().
  = All other string writing methods are handled as they call this one.
- Move json_escape_to_string() from opt_histogram_json.cc to
  my_json_writer.cc. This is needed because JSON writer unit tests do not
  link with opt_histogram_json.
- Histogram code does its own escaping (with handling for special cases).
  Add Json_writer::add_escaped_str() to accept escaped strings.
  This is not intended to be commonly used.
- opt_trace_print_expanded_query() did its own escaping for "expanded_query"
  string. It was done in MDEV-30354. Remove quoting there as otherwise it will
  be double-quoted.
Alexander Barkov
Adding a new method Type_handler::make_and_init_table_field_ex()
Arcadiy Ivanov
Restore `hp_flush_pending_blob_free()` to entry of `heap_write()`

ee463d926d8 moved the flush from entry to after `hp_write_blobs()` to
protect against a theoretical use-after-free in `UPDATE ... FOR PORTION
OF` on HEAP tables with blob columns and application-time periods.

This caused a concrete regression: deferred blob chain records are not
freed until after `hp_write_blobs()` allocates fresh continuation records
from the tail, preventing free-list reuse.  `last_allocated` grows
monotonically instead of recycling freed slots.  The
`hp_test_freelist` unit test (Tests 1-2) catches this as assertions 8,
16, 20, 24 all fail with `last_allocated` doubling on each cycle.

Move the flush back to entry of `heap_write()`, before
`next_free_record_pos()`.  This restores free-list reuse: deferred
chain records are freed and available (on the delete list or reclaimed
by `hp_shrink_tail()`) before the new blob allocation begins.

The FOR PORTION OF scenario (HEAP + blobs + application-time periods +
`UPDATE ... FOR PORTION OF SET blob_col = ...`) remains theoretical
with no test coverage and should be addressed separately, if at all,
by materializing blob data at the SQL layer before the write.
Arcadiy Ivanov
Fix stale comments, test bugs, and expand test coverage

**Comment fixes** (source):
- `hp_create.c`: fix "VARTEXT2" → "VARTEXT4", "Paclength" typo, replace
  stale 8-line `bit_start` derivation comment with accurate 3-line version
- `sql_select.cc`: fix `make_sort_key()` → `make_sort_key_part()` in
  `remove_duplicates()` comment; clarify HEAP packed-format comment
- `field.h`: fix "inc record" → "in a record" typo

**Comment fixes** (tests):
- `hp_test_key_setup-t.cc`: replace stale Phase 1 file header and function
  comment describing `key_part->length` widening with accurate blob segment
  normalization description
- `hp_test_hash-t.c`: fix swapped field names in mixed-key record layout
  comment; remove stale "hp_hashnr is static" comment; add missing
  `bit_length=2` for VARTEXT1 segment in `setup_mixed_keydef`
- `blob_big3.test`: fix "without" → "with", "dicrectly" → "directly"
- `blob_big.inc`: fix "in both runs" → "when HEAP is used"

**Test bug fixes**:
- `blob_sj_test`: change `semijoin=off` to `semijoin=on,firstmatch=off,
  loosescan=off` so the test actually exercises DuplicateWeedout SJ strategy;
  add optimizer_switch save/restore
- `blob_fallback`: replace MD5-based integrity checks with direct
  `b = repeat(...)` comparisons
- `blob_stress`: replace 4 useless `check table` (HEAP returns "not
  supported") with echo comments
- `blob_big.inc`: add save/restore for `max_sort_length` and
  `sort_buffer_size`

**Test coverage expansion** (unit tests — `hp_test_hash-t.c`):
- Add packlength 1, 3, 4 hash/comparison tests (12 assertions)
- Add blob+blob multi-segment key test (6 assertions)
- Plan: 49 → 67

**Test coverage expansion** (MTR — `blob.test`):
- INSERT ON DUPLICATE KEY UPDATE with blobs (conflict, no-conflict,
  NULL transitions)
- JSON column CRUD on MEMORY table
- LONGBLOB at uint16 `run_rec_count` split boundary (1,048,549 / 1,048,550
  / 2MB)
- Case A/B/C exact boundary blob sizes (5B, 6B, 10KB, 50KB) with
  cross-case UPDATE
- BTREE+blob rejection (both BTREE and HASH explicit blob keys rejected;
  BTREE on non-blob column with blob data works)
- Table-full error: verify no partial rows from failed inserts (row count,
  corruption check, scan count)

**Test coverage expansion** (MTR — `blob_stress.test`):
- NULL→non-NULL and non-NULL→NULL blob UPDATE operations in the 200-cycle
  stored procedure

**Build**:
- Wire `hp_test_key_setup-t.cc` into CMake build; remove Phase 1-only
  tests (`test_rebuild_key_from_group_buff_mixed`,
  `test_varchar_promoted_to_blob`); update assertions for current blob
  segment normalization; plan: 47 → 34
Sergei Petrunia
MDEV-39768: Optimizer trace should use utf8mb4

JSON is utf8mb4. Change INFORMATION_SCHEMA.OPTIMIZER_TRACE.TRACE from
using utf8mb3 to mb4.

In order to do that:
- Add m_charset into Show::Type (NULL means "use the default")
- Make Type_handler_varchar::make_schema_field() check that field
  and use the specified charset/collation.
- Add a constructor to Show::Longtext that allows to specify the charset
- Make optimizer_trace use my_charset_utf8mb4_general_ci.
- In fill_optimizer_trace_info(), indicate that the trace we've
  collected is utf8mb4

(cherry picked INTO OPTIMIZER CONTEXT REPLAY from commit a06be0288f63f520aec95982cc6f872d26c4f23c)
Monty
squash! 4888cdb69fd115986625da2a44b78a6a3898983c

Fixed that heap_prepare_hp_create_info() honors tmp_memory_table_size
Fixed memory overrun error in hp_test_hash-t
Added DBUG_ASSERT to ensure that we are not used converted heap
keys with index_read()
Sergei Petrunia
Make INFORMATION_SCHEMA.OPTIMIZER_CONTEXT.CONTEXT binary

Before, it had default I_S charset, which is utf8mb3.
One obvious problem was that JSON context data had utf8mb4.

Also, mtr --replay-server did
"select * from information_schema.optimizer_context"
with whatever connection settings were. if the test did
"SET NAMES latin1" to set its @@character_set_results, we would
get the context converted (with losses) into latin1.
This caused context replay failures.
Arcadiy Ivanov
fixup! 6e9ae08b7ad79885425652bc648f8fc56f7b1177

Fix MSVC C4244 warning: split chained assignment to avoid uint16-to-uint8 narrowing
Arcadiy Ivanov
Add test for user variable geometry assignment in GROUP_CONCAT

`Item_func_set_user_var::create_tmp_field_ex()` calls
`create_tmp_field_ex_from_handler()` directly with raw `type_handler()`,
bypassing `type_handler_for_tmp_table()`. When the argument is geometry,
`fix_fields` preserves `Type_handler_geometry` (item_func.cc:4824), so
the geometry check in `create_tmp_field_ex_from_handler()` is the only
thing preventing a `Field_geom` from being created on the GROUP_CONCAT
temp table.
Arcadiy Ivanov
fixup! 6e9ae08b7ad79885425652bc648f8fc56f7b1177

Fix 32-bit compile: use `sizeof(void*)` for blob pointer memcpy

`portable_sizeof_char_ptr` is always 8 (the record slot width), but
actual pointer size is 4 on 32-bit. The production code (`hp_hash.c`)
uses `HP_PTR_SIZE = sizeof(void*)` for memcpy of pointer values.
The test must do the same to avoid `-Werror=stringop-overflow`.
Monty
Review fixes for "Code review feedback: `hp_update.c` cleanup, test renames, style fixes"

- Optimized struct layouts in heap.h even more by moving variables
  according to size
- Replaced max_heap_table_size with tmp_memory_table_size in tests
- In ha_heap::records_in_range, removed not needed variable blob_count
- In hp_blob.cc:hp_write_run_data() replaced loop with one memcpy
- In heap_write() replaced loop with jump to avoid code duplication

Other things:
- Removed extra empty lines
- Reformatted rows to fit 80 characters
- Removed not needed braces from blocks with no varaibles.
Arcadiy Ivanov
Use SHA256 integrity checks in blob_fallback test

Store SHA2(blob, 256) on insert and compare on retrieval instead of
direct string comparison. Use non-repeat blob patterns to make
corruption detection more meaningful.
Sergei Petrunia
MDEV-39768: Optimizer trace should use utf8mb4

JSON is utf8mb4. Change INFORMATION_SCHEMA.OPTIMIZER_TRACE.TRACE from
using utf8mb3 to mb4.

In order to do that:
- Add m_charset into Show::Type (NULL means "use the default")
- Make Type_handler_varchar::make_schema_field() check that field
  and use the specified charset/collation.
- Add a constructor to Show::Longtext that allows to specify the charset
- Make optimizer_trace use my_charset_utf8mb4_general_ci.
- In fill_optimizer_trace_info(), indicate that the trace we've
  collected is utf8mb4

(cherry picked into Context Replay branch from commit 4ca87158c36635358c00c2c67653ec479e30c57d)
Arcadiy Ivanov
fixup! 07846d3ef329caca03d26e58e200f3f2d1e4f04b

Replace exact `Created_tmp_%` counts with ON/OFF checks in blob_big tests

Exact temp table counts vary between 32-bit and 64-bit platforms
because view protocol creates additional temp tables whose overflow
behavior depends on pointer size and internal structure sizes.

Replace `SHOW STATUS LIKE 'Created_tmp%'` with a query that reports
ON/OFF for `CREATED_TMP_TABLES` and `CREATED_TMP_DISK_TABLES`. This
preserves the key assertions (disk tables created vs not created)
while being platform-independent.
Sergei Petrunia
MDEV-39720: Optimizer Trace doesn't quote string values, may have invalid JSON

- Make Json_writer::add_str() do the quoting with json_escape_to_string().
  = All other string writing methods are handled as they call this one.
- Move json_escape_to_string() from opt_histogram_json.cc to
  my_json_writer.cc. This is needed because JSON writer unit tests do not
  link with opt_histogram_json.
- Histogram code does its own escaping (with handling for special cases).
  Add Json_writer::add_escaped_str() to accept escaped strings.
  This is not intended to be commonly used.
- opt_trace_print_expanded_query() did its own escaping for "expanded_query"
  string. It was done in MDEV-30354. Remove quoting there as otherwise it will
  be double-quoted.
Arcadiy Ivanov
Add missing `hp_shrink_tail()` after immediate blob chain frees

`hp_free_run_chain()` puts freed continuation records onto the delete
list but does not reclaim tail-positioned records. `hp_shrink_tail()`
must be called afterward to shrink `block.last_allocated` when freed
records happen to be at the tail, preventing monotonic HP_BLOCK growth.

The deferred-free path (`hp_flush_pending_blob_free_impl`) already
calls `hp_shrink_tail()`, but three immediate-free sites did not:

- `hp_free_blobs()`: used by DELETE on internal temporary tables
- `heap_update()` internal-table path: immediate free of old chains
- `heap_update()` rollback path: free of partially-written new chains

Expose `hp_shrink_tail()` in `heapdef.h` (was `static` in `hp_blob.c`)
and add the call at all three sites.
Sergei Petrunia
MDEV-39720: Optimizer Trace doesn't quote string values, may have invalid JSON

- Make Json_writer::add_str() do the quoting with json_escape_to_string().
  = All other string writing methods are handled as they call this one.
- Move json_escape_to_string() from opt_histogram_json.cc to
  my_json_writer.cc. This is needed because JSON writer unit tests do not
  link with opt_histogram_json.
- Histogram code does its own escaping (with handling for special cases).
  Add Json_writer::add_escaped_str() to accept escaped strings.
  This is not intended to be commonly used.
- opt_trace_print_expanded_query() did its own escaping for "expanded_query"
  string. It was done in MDEV-30354. Remove quoting there as otherwise it will
  be double-quoted.
Arcadiy Ivanov
Fix CI failures: 32-bit/big-endian test portability and non-deterministic ordering

- `hp_test_helpers.h`: use `sizeof(blob_data)` instead of `portable_sizeof_char_ptr`
  for `memcpy` of pointer values, fixing `-Werror=stringop-overread` on 32-bit
- `hp_test_hash-t.c`: use `int2store()` for blob length (endian-safe) fixing
  s390x big-endian failures in DISTINCT/GROUP BY key tests; use
  `portable_sizeof_char_ptr` for blob+blob key offset (32-bit blob2 length read);
  use `sizeof(varchar_data)` for pointer copies
- `heap.blob_big2`: reduce `tmp_memory_table_size` 65536→32768 so UNION DISTINCT
  overflows to disk on 32-bit (smaller recbuffer = more rows fit in memory)
- `main.type_bit`: add `--disable_cursor_protocol` around metadata-sensitive
  DISTINCT/GROUP BY queries (cursor mode loses `UNIQUE_KEY_FLAG` on temp table)
- mroonga fulltext_order tests: add `, title` tiebreaker to ORDER BY for
  equal MATCH scores; mask `Created_tmp_tables` counter with `--replace_column`
Monty
Remove pack_length_no_ptr()

Replaced pack_length_no_ptr() with existing length_size() that does the
same thing.
Arcadiy Ivanov
fixup! 45718801fb0755ea2092d8382bf29f7490667dcb

Disable ps_protocol in blob_update_overflow status checks

PS protocol creates extra internal temp tables, inflating
Created_tmp_tables counters. Disable all protocol variants
around FLUSH STATUS / SHOW STATUS sections.