Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Sergei Golubchik
Merge https://github.com/drrtuy/duckdb-engine into 11.4
drrtuy
chore(): avoid compilation error for the fiber test TU.
drrtuy
chore(test): fix flacky cross_join_where test.
drrtuy
fix(cej): escape column names, remove bin objects from the repo, add tests.
Rex Johnston
MDEV-39495 Parallel Query: Use temporary work tables to ship results: basic test

Built on top of MDEV-39492, the parallel worker manager creates N
temporary tables and gives them to each parallel worker to populate.
At the conclusion of all worker threads, each row from the above tables
is added to the join_tab->table representing the query result.  To use

MariaDB [test]> set session parallel_worker_threads=10;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> select SQL_BUFFER_RESULT * from t1 join seq_1_to_5 on t1.a = seq;
+------+------+-----+
| a    | b    | seq |
+------+------+-----+
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
|    1 | one  |  1 |
|    2 | two  |  2 |
|    1 | un  |  1 |
+------+------+-----+
30 rows in set (0.010 sec)

Each worker has read the first table with a JT_ALL access method and
replicated the results back to the manager thread, which executes the
rest of the join.  In the above example there are 10 worker threads, so
10 copies of the join result.
LaGrunge
feat(compat): add mid() UDF, CONVERT→CAST rewrite in pushdown

- mid(str, pos[, len]) via SQL macro delegating to DuckDB substr()
  (handles multibyte UTF-8 correctly)
- CONVERT(expr, TYPE) → CAST(expr AS TYPE) SQL rewrite in pushdown
  handler for MariaDB-specific syntax
- Remove mid SQL macros from duckdb_manager.cc

Tests progress:
- duckdb_string_func: past MID (line 212) to NOT REGEXP (line 226)
- duckdb_fix_sql: past CONVERT to DuckDB CAST type limitation
LaGrunge
feat(compat): curdate/curtime macros, STRAIGHT_JOIN/index hints/LIMIT
rewrites, enable duckdb_sql_syntax test

- curdate()/curtime() macros → current_date/current_time
- STRAIGHT_JOIN → CROSS JOIN rewrite (before conditionless JOIN scan)
- Strip FORCE/USE/IGNORE INDEX(...) hints
- LIMIT offset,count → LIMIT count OFFSET offset
- Strip SQL_CALC_FOUND_ROWS and other SELECT hints
- addtime/subtime TIME overload: wrap around 24h for DuckDB range

Enable duckdb_sql_syntax test. Result diff: only ROLLUP grand total
row on empty table (+NULL,NULL,NULL — DuckDB SQL standard behavior)
and removed deprecation warnings.

Enabled: 23 -> 24.
LaGrunge
feat(compat): CURRENT_TIME(N)/CURRENT_DATE()/CURRENT_TIMESTAMP() rewrite,
curtime(fsp), datediff(d1,d2) macro

- Strip parens and args from CURRENT_TIME(N), CURRENT_DATE(),
  CURRENT_TIMESTAMP() — DuckDB uses keyword form without parens
- curtime(fsp) macro: accept optional fractional seconds precision arg
- datediff(d1, d2) macro: d1::DATE - d2::DATE (MariaDB 2-arg form)

duckdb_time_func progresses: line 45 → 155 (of 847).
Next blocker: DATEDIFF(TIMESTAMP, TIME) — DuckDB can't cast TIME→DATE.
drrtuy
feat(runtime): fiber runtime based on ASM borrowed from libmariadb + custom select_result_interceptor for DuckDB.
drrtuy
chore(build) preparation for code refactoring.
Sergei Golubchik
rpm
Sergei Golubchik
Merge https://github.com/mariadb/duckdb-engine into 11.4
Sergei Golubchik
add submodule, most-merge fixes
LaGrunge
feat(compat): override length(VARCHAR) and ascii(VARCHAR) for MariaDB
byte semantics, enable duckdb_string_func test

Add C++ UDF overloads via ALTER_ON_CONFLICT:
- length(VARCHAR) → byte count (was: DuckDB char count)
- ascii(VARCHAR) → first byte value (was: DuckDB Unicode codepoint)

This matches MariaDB LENGTH()/ASCII() semantics for pushdown queries.

Result diff reviewed:
- Warnings removed (DuckDB doesn't emit truncation warnings)
- REGEXP_LIKE/multi-arg REGEXP_* sections removed (MySQL 8.0 only)
- ORD() for multibyte: 15111600→25968 (MariaDB multibyte ORD vs
  DuckDB Unicode codepoint — minor, TODO)

Enabled: 22 -> 23.
drrtuy
chore(build): refactor code breaking it into separate modules.
drrtuy
chore(): docs updates based on the recent changes.
Sergei Petrunia
--disable_replay testfile in  main.invisible_field_debug test
LaGrunge
feat(pushdown): add CROSS JOIN, REGEXP, NOT REGEXP rewrites + mid UDF

SQL rewrites in pushdown handler:
- Conditionless JOIN → CROSS JOIN (scan for ON/USING before boundary)
- expr REGEXP pattern → expr ~ pattern (DuckDB regex operator)
- expr NOT REGEXP pattern → expr !~ pattern

mid(str, pos[, len]) implemented as SQL macro delegating to DuckDB
substr() for proper UTF-8 multibyte handling.

Tests progress:
- duckdb_sql_syntax: past JOIN to HIGH_PRIORITY (line 34)
- duckdb_string_func: past REGEXP to REGEXP_INSTR (line 302)
Rex Johnston
MDEV-39492 Parallel Query: Study how to create worker threads

Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.

2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name).  Main management
class created in the stack in JOIN::exec, implemented for the
top level select.

Current parallel_worker_thread_func sleeps for 10 seconds, generates
a warning, signals the main thread, sleeps 10 seconds, signals the
main thread again, sets it's finished flag and cleans it's THD.

The main thread loops through worker threads, looking for finished
thread and cleans them up if they have finished.
It then waits for a signal, then processes it's message queue.

Threads are registed in server_threads, so are visible in
information_schema.processlist and the show processlist command.

We check that a kill query on a parallel worker is passed onto it's
manager and the query is properly aborted, and that a kill connection
is handled properly in parallel_worker.test.

Review input 1: cleanup earlier

Do cleanup before we've finished sending the result to the client.
This way, one can see the errors (and eventually warnings) marshalled
back to the main thread and returned to the user:

MariaDB [test]> set parallel_worker_threads=10;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> select seq from seq_1_to_10;
ERROR 4103 (HY000): Argument to the worker_busted_function() function does not belong to the range [0,1]

Assisted by Sergei Petrunia and Claude Code.
Sergei Golubchik
improve ExternalProject

* don't bother merging libraries
* gitignore the build dir
* prefix dir = build dir
* produce the progress log while building
Oleg Smirnov
Fix failing tests
Sergei Petrunia
Fix --replay-server for main.derived_view
drrtuy
chore(): update documentation.
drrtuy
chore(build): install build dependencies in build.sh.
LaGrunge
feat(compat): implement regexp_instr, regexp_replace, regexp_substr,
json_unquote using RE2 directly

Instead of calling MariaDB Item_func_* classes (which require
current_thd unavailable in DuckDB worker threads), implement
regex functions using DuckDB's bundled RE2 library directly:
- regexp_instr(str, pattern) → position of first match
- regexp_replace(str, pattern, repl) → RE2::GlobalReplace
- regexp_substr(str, pattern) → first matching substring
- json_unquote(str) → strip JSON quotes and unescape

Add DuckDB third_party/re2 to include path in cmake.

duckdb_string_func progresses past REGEXP_INSTR to test issue
(MariaDB 11.4 doesn't support 3-arg REGEXP_INSTR).
drrtuy
fix(pushdown): skip-list of all escaped strings to avoid SQL expressions lookup in those.
drrtuy
Expose static symbols leveraged in DuckDB to stringify WHERE conditions for other MariaDB engines.
LaGrunge
feat(compat): addtime/subtime C++ UDFs, convert_tz macro, LIMIT and
SELECT hints rewrite

- addtime/subtime: C++ UDFs parsing MariaDB interval format
  'D H:M:S.us', with TIMESTAMP and TIME overloads (TIME wraps 24h)
- convert_tz(ts, from, to): macro via DuckDB timezone()
- Remove MariaDB SELECT hints: HIGH_PRIORITY, SQL_NO_CACHE,
  SQL_SMALL_RESULT, SQL_BIG_RESULT, SQL_CALC_FOUND_ROWS, STRAIGHT_JOIN
- Rewrite LIMIT offset,count → LIMIT count OFFSET offset

Tests progress:
- duckdb_time_func: past addtime to CONVERT_TZ (line 60)
- duckdb_sql_syntax: past hints to LIMIT syntax (line 47)
Sergei Golubchik
don't modify server's error messages
drrtuy
fix(build): now DuckDB core extensions are statically linked instead auto-loaded.
LaGrunge
fix(rebase issues): Some tests are broken now, a big thanx
LaGrunge
feat(compat): port full hex/oct/bin from AliSQL, add locate C++ UDF

Port production-quality hex/oct/bin scalar functions from AliSQL DuckDB
fork. Supports all numeric types including HUGEINT (no precision loss
for large decimals), VARCHAR (parse as double), DOUBLE/FLOAT (round
first), BLOB (hex encoding). Replaces previous SQL macros.

Add locate(needle, haystack) and locate(needle, haystack, pos) as
native C++ scalar functions with proper 2-arg and 3-arg overloads.

Remove oct/bin/locate SQL macros from duckdb_manager.cc — now handled
by C++ UDFs in duckdb_mysql_compat.cc.

Tests progress: duckdb_string_func past LOCATE/BIN/HEX to MID (line
212), duckdb_fix_sql past hex(decimal) to CONVERT (line 53).
Oleg Smirnov
WIP: Integrate chunked InnoDB scan with parallel workers framework
drrtuy
fix(cej): WHERE predicate now does not contain alias or table name in column idents.
drrtuy
fix(mtr): trim couple tests to pass on U24.04.
drrtuy
fix(pushdown): fix for STRAIGHT_JOIN, various JOIN re-write cases and also recursive cases WITH ROLLUP and LIMIT clauses.
Sergei Petrunia
Fix replay for non-default @@sql_mode settings.

sql_mode=ORACLE and similar settings may prevent the table and/or
column options from being printed correctly.

Remove them when calling show_create_table_ex().

Since the output of show_create_table_ex() is added into the
SQL script, also add a "SET sql_mode=..." before the CREATE TABLE
and another one to restore the value after it.

This fixes main.intersect_all
drrtuy
fix(build):thread local maps with default TLS model got broken with MSAN build.
LaGrunge
feat(compat): add strcmp, substring_index, to_base64 macros, RLIKE
rewrite, adapt test for MariaDB 11.4

New SQL macros: strcmp(a,b), substring_index(s,d,c), to_base64(x).
Add RLIKE/NOT RLIKE → ~/!~ rewrite alongside REGEXP.

Adapt duckdb_string_func test:
- Comment out REGEXP_LIKE section (MySQL 8.0 only)
- Comment out multi-arg REGEXP_INSTR/REPLACE/SUBSTR (MySQL 8.0 only)

Test runs to completion but has semantic differences:
- LENGTH() returns chars in DuckDB vs bytes in MariaDB
- ASCII() returns codepoint in DuckDB vs first byte in MariaDB
Needs LENGTH/ASCII UDF overloads for byte semantics.
drrtuy
feat(cej): MDB tables scan now uses mysql_execute_command to to leverage optimizer and various access paths for a scan.