Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Dave Gosselin
MDEV-35747: Wrong result from prepared TVC with parameter markers

The setup of column type information in table_value_constr::prepare()
was wrapped in an "if (!holders)" guard so that it runs only once per
prepared statement.  However, the guard was too wide because it bound
the allocation of item holders (which should happen only once) to the
collection of type information (which should happen on each
execution).

This leaves the TVC stuck with whatever placeholder type the parameter
had at PREPARE time which may not match the type of the next
substitution.  A type holder has no actual type at PREPARE time.  Its
type only becomes known when a value is bound at EXECUTE time.  So
both the TVC types and the corresponding Item_type_holder instance in
the SELECT item list must be computed again on every EXECUTE.

This patch separates the work done once per prepared statement from
the work done on every execution as implied above.
Monty
Review commit of Avoid double blob materialization
ParadoxV5
MDEV-37321: Move select IO thread steps and select connector functions

To prepare for MDEV-37321, this commit moves them to a
`Remote_event_stream` class outside of `sql/`.
The `.cc` file will switch to Connector/C unbeknownst to `rpl_mi.cc`.

Only the following steps are reörganized to reusable instance methods:
* Connection setup and startup
* VIO management
  * Abort when the thread is killed
  * Closing
    (already covered by Connector/C, but not by the current `libmysql`)

The following steps are currently not öptimal for moving
(have verbose changesets), so they are only wrapped, not reörganized:
* steps that run queries *with copy-pasted code*
* sending commands `COM_REGISTER_SLAVE` & `COM_BINLOG_DUMP`,
  including serializing their arguments
  * Connector/C does not have a formal API for serializing
    command arguments, especially variable-length strings.
  * Connector/C has a Binlog API that duplicates this step, but that
    API is not modular for the Server’s full replication capabilities.
* error checking

These require refactors and ideally an evaluation for `mariadb-binlog`.

To enable this migration, this commit additionally:
* Reserves the management of the IO thread’s VIO to the client library
* Draft-specific: omits MDEV-28114’s kludge failure
  recovery for Semi-Sync graceful self-KILL since
  * MDEV-32551 fixed the root bug on the ACK receiver, and
  * MDEV-39583 plans to remove this entire sub-component.

Draft-specific: Until Connector/C exports the pre-decompress (network)
byte count, `@@read_binlog_speed_limit` will measure the post-decompress
count even in `@@slave_compressed_protocol` for now.
Sergei Golubchik
MDEV-39582 mariadb-binlog missing sanity checks for ibb files

check that binlog_page_size <= BINLOG_PAGE_SIZE_MAX
Kristian Nielsen
MDEV-39582: mariadb-binlog missing sanity checks for ibb files

The binlog page size is currently fixed at 16k. Don't attempt to read
using a different page size found in the file header, the code doesn't
support it.

Signed-off-by: Kristian Nielsen <[email protected]>
bsrikanth-mariadb
MDEV-39405: store the necessary plugin-engines optimizer costs

store the necessary plugin-engines optimizer costs that are used by the query
into the context, so that the replay server uses the same while computing
query cost
Monty
Review of reclaim tail records on failed blob allocation

- Simplify loop in hp_shrink_tail to avoid multiplcations
- Moved assert from callers of hp_shrink_tail() to hp_shrink_tail()
Monty
Review fixes for "Batch tail allocation for blob continuation chains"

- Optimized setting is_only_run (avoiding one/two if's for each loop)
- Removed test for run_rec_count == 1 in a context where it was guaranteed
  to be 1.
- Removed not needed variable prev_pos (as it was always identical to
  run_start). This also fixes one of the bugs fixed by the next commit.
Dave Gosselin
MDEV-35747: Wrong result from prepared TVC with parameter markers

The setup of column type information in table_value_constr::prepare()
was wrapped in an "if (!holders)" guard so that it runs only once per
prepared statement.  However, the guard was too wide because it bound
the allocation of item holders (which should happen only once) to the
collection of type information (which should happen on each
execution).

This leaves the TVC stuck with whatever placeholder type the parameter
had at PREPARE time which may not match the type of the next
substitution.  A type holder has no actual type at PREPARE time.  Its
type only becomes known when a value is bound at EXECUTE time.  So
both the TVC types and the corresponding Item_type_holder instance in
the SELECT item list must be computed again on every EXECUTE.

This patch separates the work done once per prepared statement from
the work done on every execution as implied above.
bsrikanth-mariadb
MDEV-39405: store the necessary plugin-engines optimizer costs

store the necessary plugin-engines optimizer costs that are used by the query
into the context, so that the replay server uses the same while computing
query cost
Alexey Botchkov
MDEV-39536 ASAN heap-buffer-overflow in process_str_arg upon ER_TRUNCATED_WRONG_VALUE "Incorrect XML value".

Test casse added.
Rex Johnston
MDEV-39492 Parallel Query: Study how to create worker threads

Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.

2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name).  Main management
class created in the stack in JOIN::exec, implemented for the
top level select.

Current parallel_worker_thread_func sleeps for a random time, generates
a warning, signals the main thread, sleeps a random time, signals the
main thread again, sets it's finished flag and cleans it's THD.

The main thread loops through worker threads, looking for finished
thread and cleans them up if they have finished.
It then waits for a signal, then processes it's message queue.

The thread management data is allocated on the stack in JOIN::exec.
Everything else is allocated using my_malloc() and my_free().
Kristian Nielsen
MDEV-39604: binlog_checksum should be ignored in new binlog

Don't attempt to rotate any old-style binlog files if trying to set
@@binlog_checksum. The @@binlog_checksum variable is unused with the
new binlog, and there are no old-style binlog files to rotate.

Signed-off-by: Kristian Nielsen <[email protected]>
Kristian Nielsen
MDEV-39604: binlog_checksum should be ignored in new binlog

Don't attempt to rotate any old-style binlog files if trying to set
@@binlog_checksum. The @@binlog_checksum variable is unused with the
new binlog, and there are no old-style binlog files to rotate.

Signed-off-by: Kristian Nielsen <[email protected]>
Dave Gosselin
MDEV-38508: Constant table detection

If a table that's in a FULL OUTER JOIN is found to be a const
table, then don't allow the constant table optimization to
take place.

Later, when we support FULL OUTER JOIN on the inner side of
other join types then we may be able to relax this restriction.
Alexander Barkov
MDEV-39587 Package-wide TYPE for variable declarations

Note: only sql_mode=ORACLE. Because TYPE is only available in sql_mode=ORACLE.

TODO: tests: all TYPE grammar
TODO: inside the package itself
Alexey Botchkov
MDEV-39589 Wrong results with xmltype as a virtual column.

Store value in the XML column even if an error happened.
Kristian Nielsen
MDEV-39582: mariadb-binlog missing sanity checks for ibb files

Validate the chunk_len in the binlog chunk reader, so we don't try to read
data outside of the page.

Signed-off-by: Kristian Nielsen <[email protected]>
Alexey Botchkov
MDEV-39575 Protocol::end_statement(): Assertion `0' failed after insert into xmltype union of valid and invalid xml.

Field_xmltype::store returns 0 as we issue warning in case of the
incorrect XML value.
Kristian Nielsen
MDEV-39582: mariadb-binlog missing sanity checks for ibb files

Validate the chunk_len in the binlog chunk reader, so we don't try to read
data outside of the page.

Signed-off-by: Kristian Nielsen <[email protected]>
Dave Gosselin
Remove the unused AGGR_OP::flush_record method
Thirunarayanan Balathandayuthapani
MDEV-28730 Remove internal parser usage from InnoDB FTS

Introduce QueryExecutor to perform direct InnoDB record scans
with a callback interface and consistent-read handling.
Also handles basic DML operation on clustered index of the table

Newly Added file row0query.h & row0query.cc

QueryExecutor class the following apis
  read(): iterate clustered index with RecordCallback
  read_by_index(): scan secondary index and fetch clustered row
  lookup_clustered_record(): resolve PK from secondary rec
  process_record_with_mvcc(): build version via read view
                              and skip deletes
  insert_record(): Insert tuple into table's clustered index
  select_for_update(): Lock the record which matches with search_tuple
  update_record():  Update the currently selected and X-locked
                    clustered record.
  delete_record():  Delete the clustered record identified by tuple
  delete_all(): Delete all clustered records in the table
  replace_record(): Tries update via select_for_update() +
  update_record(); if not found, runs insert_record

read(), read_by_index(): Offset computation happens only
when there is need to construct the previous version of
the record or when there is a clustered index lookup

Add FTSQueryExecutor class as a thin abstraction over QueryExecutor.
This class takes care of open, lock, read, insert, delete
for all auxiliary tables INDEX_[1..6], common FTS tables
(DELETED, DELETED_CACHE, BEING_DELETED, CONFIG..)

FTSQueryExecutor Class which has the following function:
  Auxiliary table functions : insert_aux_record(),
                              delete_aux_record(),
                              read_aux(), read_aux_all()

  FTS common table functions : insert_common_record(),
                              delete_common_record(),
                              delete_all_common_records(),
                              read_all_common()

  FTS CONFIG table functions : insert_config_record(),
                              update_config_record(),
                              delete_config_record(),
                              read_config_with_lock()

Introduce CommonTableReader callback to collect doc_id_t from
fulltext common tables (DELETED, BEING_DELETED, DELETED_CACHE,
BEING_DELETED_CACHE). These table share the same schema strucutre.

extract_common_fields(): Extract the common table fields

Introduce ConfigReader callback to extract key, value from
fulltext config common table (CONFIG). This table has
<key, value> schema

extract_config_fields(): Reads the config table fields
from the record without creating offsets

Introduce AuxCompareMode and AuxRecordReader to scan FTS auxiliary
indexes with compare+process callbacks.

extract_aux_fields() : extracts the fields of the auxiliary table
from the record without creating offsets

Removed fts0sql.cc file, commented and unused fts funtions

Removed fts_table_t usage from fts_query_t and fts_optimize_t

fts_write_node(): Refined to use FTSQueryExecutor and fts_aux_data_t.
Change fts_select_index{,_by_range,_by_hash} return type
from ulint to uint8_t and simplify return flow.

fts_query(): fts_query code now uses
QueryExecutor::read(), read_by_index() with RecordCallback

fts_optimize_write_word(): To delete (or) insert via
FTSQueryExecutor::delete_aux_record()/insert_aux_record()
using fts_aux_data_t;

fts_optimize_table() : Assigns thd to transaction even it is
called via user_thread or fulltext optimize thread.

InnoDB tries to acquire all auxiliary table and common table
during fts_sync_table(), fts_optimize_table() and fts_query()
at once. This optimization avoids repetitively acquiring
dict_sys.latch
Alexander Barkov
MDEV-39587 Package-wide TYPE for variable declarations

Note: only sql_mode=ORACLE. Because TYPE is only available in sql_mode=ORACLE.

TODO: tests: all TYPE grammar
TODO: sql_path resolution
TODO: inside the package itself
Rex Johnston
MDEV-39492 Parallel Query: Study how to create worker threads

Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.

2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name).  Main management
class created in the stack in JOIN::exec, implemented for the
top level select.

Current parallel_worker_thread_func sleeps for a random time, generates
a warning, signals the main thread, sleeps a random time, signals the
main thread again, sets it's finished flag and cleans it's THD.

The main thread loops through worker threads, looking for finished
thread and cleans them up if they have finished.
It then waits for a signal, then processes it's message queue.

The thread management data is allocated on the stack in JOIN::exec.
Everything else is allocated using my_malloc() and my_free().
Alexander Barkov
MDEV-39587 Package-wide TYPE for variable declarations

Note: only sql_mode=ORACLE. Because TYPE is only available in sql_mode=ORACLE.

TODO: tests: all TYPE grammar
TODO: inside the package itself
bsrikanth-mariadb
MDEV-39405: store the necessary plugin-engines optimizer costs

store the necessary plugin-engines optimizer costs that are used by the query
into the context, so that the replay server uses the same while computing
query cost
Monty
Review of "Replace `hp_blob_run_format()` enum with direct bit testing"

Change 'uchar format' to 'uint8 format' to make it clear that we use
format as a number.
Daniel Black
MDEV-30518: JSON functions cannot be killed on big-endian

The json_engine killed_ptr was of type uchar however the enum
in the server is dependant on the architecture. On big-endian
architectures like IBM Z, retreiving a uchar* retreived the
unmodifed part of the THD->kill enum location and was always 0.

As C++11 allows enums to inherit a type, used uint32_t as the base
class of killed_state type in the server and used uint32_t in the
json library.

reload_acl_and_cache required an expression change to avoid
the compile error:
sql/sql_reload.cc:472:24: error: enumerated and non-enumerated
type in conditional expression [-Werror=enum-conversion]
  472 |  return result || (thd ? thd->killed : 0);

Added the kill_ptr assignment the following functions can be
killed:
* Item_func_json_depth::val_int
* Item_func_json_type::val_str
* Item_func_json_length::val_int

Item_func_json_array_append::val_str, check_killed() only applied
to json_error: label and not return_null.

Item_func_json_overlaps::val_bool ensure check_killed() is called.

Item_func_json_format::val_str(), corresponding to the SQL,
json_compact, json_detailed, json_loose - add debug instrumentation
for the func_json_notembedded test.

MDEV-26726 (fcd345de485f) added a json_pause_execution sync point
however the test case used debug_max_statement_time, which doesn't
exist. Fix the sync point name in the test.
Monty
Review commit for Overflow-to-Aria on `ha_update_tmp_row()`...

- Updated test to print number created temporary files for all
  test parts.
- Fixed long comment in hp_blob.c
Kristian Nielsen
MDEV-39582: mariadb-binlog missing sanity checks for ibb files

The binlog page size is currently fixed at 16k. Don't attempt to read
using a different page size found in the file header, the code doesn't
support it.

Signed-off-by: Kristian Nielsen <[email protected]>
Alexey Botchkov
MDEV-39575 Protocol::end_statement(): Assertion `0' failed after insert into xmltype union of valid and invalid xml.

Field_xmltype::store has to do the my_error when it gets invalid XML,
  just the warning with the -1 return results in the DBUG_ASSERT.

Also the UpdateXML function result reverted back to the STRING as
it can legally produce invalid XML, so we can't have columns built
from the UpdateXML to be XMLTYPE.
Alexey Botchkov
MDEV-39573 Server crashes in Item_char_typecast::fix_length_and_dec_internal after select xmltype.

Charset specified.
Dave Gosselin
MDEV-35747: Wrong result from prepared TVC with parameter markers

The setup of column type information in table_value_constr::prepare()
was wrapped in an "if (!holders)" guard so that it runs only once per
prepared statement.  However, the guard was too wide because it bound
the allocation of item holders (which should happen only once) to the
collection of type information (which should happen on each
execution).

This leaves the TVC stuck with whatever placeholder type the parameter
had at PREPARE time which may not match the type of the next
substitution.  An Item_param has no actual type at PREPARE time.  Its
type only becomes known when a value is bound at EXECUTE time.  So
both the TVC types and the corresponding Item_type_holder instance in
the SELECT item list must be computed again on every EXECUTE.

This patch separates the work done once per prepared statement from
the work done on every execution.

A separate change in Item_args::alloc_arguments resets arg_count to
zero on entry, so that running get_type_attributes_for_tvc again on
persistent Type_holder slots does not accumulate arguments across
executions.
Dave Gosselin
MDEV-39569: Skip FULL JOIN rewrite to inner side of an outer join

Prevent simplify_joins from rewriting a chained FULL JOIN into a query
where a FULL JOIN could end up on the inner side of another outer
join.  Of course, this means that we will have a null complement pass
that the rewritten query would have avoided.  Once we support FULL
JOINs on the inner side of outer joins, in phase 3, then we can relax
this constraint.