Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Nikita Malyavin
MDEV-37850 Wrong error on DROP TABLE GTT after DROP DATABASE

Turns out that temporary tables are not dropped as part of DROP
DATABASE.

This patch implements dropping child global temporary tables for the
currently dropping database.

That is, if the child table was opened in this connection, it will be
dropped together with the parent table.

If another connection attempts to drop the database, it will wait while
the table is in use.
Kristian Nielsen
Binlog-in-engine: Fix inconsistency in stmt/trx cache use

There was a bug when a DML used both the stmt and trx cache but the trx
cache is empty (in this case INSERT IGNORE that inserts nothing but still
allocates from a sequence). The code would put the GTID in the stmt cache
but binlog the trx cache with some stray dangling annotate/tablemap event.
Thus a completely invalid event group resulted.

In general, the logic for selecting stmt or trx cache was complicated and
inconsistent in multiple places around the code. This patch improves by
introducing a single binlog_cache_mngr::engine_cache_data() which holds the
logic and is used in all places.

This patch also fixes a few problems with the GTID event binlogged for user
XA, found during debugging.

Signed-off-by: Kristian Nielsen <[email protected]>
Nikita Malyavin
MDEV-37694 ASAN heap-use-after-free in check_column_name on CoR ... LIKE

fixed by "Rework how global temporary tables are dropped"
Nikita Malyavin
MDEV-37667 SIGSEGV on ALTER on locked GTT in low memory env

Alter table, as one of its stages, creates a temporary table from a
given table. If something goes wrong, this table will be freed.

We never distinguished a child table of global temporary table and a
temporary table created from parent for internal purposes, as this one:
They are even added to the temporary_tables list as a normal tmp table.

This commit denotes a difference for global temporary tables as with
share->mdl_request.ticket set to NULL.

Fix:
If it's a temporary table created from global temporary table for
internal purposes,
1. do not free mdl_ticket
2. do not alter global_temporary_tables_count
Nikita Malyavin
MDEV-37693 use-after-free in mysql_ha_flush after 2nd HANDLER OPEN execution

An error arises due to early deny after open_table in mysq_admin_table,
taken place in-between two handler opens, preceeding by SELECT in an
open transaction.

Analysis:
The crash happens when another mysql_ha_flush call (for example, in the
beginning of SELECT) writes table->mdl_ticket, which wasn't cleaned up
after release.
The ticket release happens normally, at mysql_ha_close_table call. This
could be another HANDLER CLOSE call, but it happens implicitly by LOAD
INDEX.
There are certain circumstances, under which table->mdl_ticket is
accessed by the handler:
  if (table_list->mdl_request.ticket &&
      thd->mdl_context.has_lock(mdl_savepoint, table_list->mdl_request.ticket))
in mysql_ha_open,
meaning that the lock should have had existed in the transaction during
open.
This condition is true for the first HANDLER OPEN, because of
preceeding SELECT * FROM t, which is supposed to hold a lock during the
whole transaction.
However, killed LOAD INDEX releases that lock, resulting in skipping a
write into table_list->table->mdl_ticket durind a second HANDLER OPEN.

This doesn't happen for local temporary tables, because they don't hold
table_list->mdl_request.ticket.
This doesn't happen for normal tables, because table->mdl_ticket is
cleaned up during close_thread_table.

Solution:
Clean up table->mdl_ticket on mysql_ha_close_table, as that is the only
case when it's supposed to be set.
Nikita Malyavin
MDEV-37871 SIGSEGV in I_P_List_iterator on DROP DATABASE test

Happens on dropping the database on a newly created conneciton.

temporary_tables can be NULL in THD::global_tmp_drop_database.
Nikita Malyavin
MDEV-37718 Assertion '!thd->rgi_slave' failed on GTT DML

Assertion '!thd->rgi_slave' failed in open_global_temporary_table on
CREATE/ANALYZE GTT, also a SIGSEGV in the release build.

This is a result of a fact that some operations led to opening a child
table on slave. The bug can be split in two parts:

1. SELECT part of CREATE...SELECT is replicated. It was binlogged,
despite table_creation_was_logged=0 explicitly set. To avoid, fall to
the row logging path of create_select, i.e. log SHOW CREATE output, but
don't actually log rows.
The relevant changes are in sql_insert.cc

2. Admin commands like ANALYZE TABLE still create a child table on open,
but are binlogged. Binlogging them would be otherwise harmless, but
better to avoid it, until the commands are fully supported and make
sense.
For now, avoid binlogging them with lex->no_write_to_binlog=false.
Nikita Malyavin
MDEV-37817 DROP TABLE GTT doesn't succeed for ON COMMIT DELETE ROWS

Drop child GTTs on pre-statement impllicit commit stage.

We can't do that inside trans_commit_implicit, because some implicit
commits are fake and part of a bigger logic, where we'd want to preserve
open GTTs.

We can't call mark_tmp_tables_as_free_for_reuse either, since there
could be tables just opened for this transaction, so only drop
exactly what's needed, assuming that tables were marked for reuse in
prev statement, but not dropped yet. One place where mark for reuse is
still required is after closing HANDLER tables.
Marko Mäkelä
Merge 10.11 into 11.4
Marko Mäkelä
MDEV-37860: Assertion !waiting failed with thread-handling=pool-of-threads

In commit 13076351f10f59a9bedb32e5233666d35c86f51f (MDEV-37152)
some code was refactored that would break the server-side
connection pool logic. Calls to thd_wait_begin() and thd_wait_end()
were not balanced.

buf_read_page_low(): Only invoke thd_wait_begin() and thd_wait_end()
for synchronous reads, and do that also when the read operation fails.

Reviewed by: Vladislav Lesin
Nikita Malyavin
MDEV-37666 Global temporary table can be created w/ versioning using CREATE LIKE

Turns out, alter_info->flags is not set by mysql_prepare_alter_table.
* Use create_info->options instead.

The same problem affects foreign keys. There is no single field to
check, so we'd need to loop through the keys in this case.
The same is already done for partitioning.
* Extraxt the key check-loop into Alter_info::has_foreign_keys
* Use it for partitions and global temporary tables check
Marko Mäkelä
Merge 10.6 into 10.11
Teemu Ollakka
MDEV-34493 memory leak when using galera node with only replication thread

Applying of write set containing many row events causes
applier `thd->mem_root` to grow until applying is over.
This may contribute to memory fragmentation problem, producing
large chunks of memory which are not released back to operating
system.

In order to reduce memory fragmentation, free `thd->mem_root`
in `Rows_log_event::do_apply_event()` after applying a row event
with `F_STMT_END` set.
Sam Bull
MDEV-36009: Systemd: Restart on OOM

Per https://github.com/systemd/systemd/issues/36529 OOM counts
as a on-abnormal condition. To ensure that MariaDB testart on
OOM the Restart is changes to on-abnormal which an extension
on the current on-abort condition.
Yuchen Pei
MDEV-37568 Fix remote table key direction in spider/bugfix.index

Ideally spider should throw an error when local and remote tables have
conflict definitions.

This fixes the test for --view-protocol and
--mysqld=--loose-spider-disable-group-by-handler
Dave Gosselin
MDEV-36125 Cleanup ahead of [NO_]INDEX_MERGE

Introduces code cleanup and small refactorings, such as:

  - Flip keys_to_use.is_clear_all() and de-dent
  - Rename best_trp to best_read_plan
  - Fix key_parts assignment spacing
  - find_shortest_key now keeps happy path to the left
  - Cleanup in the get_best_ror_intersect, remove_nonrange_trees,
    and restore_nonrange_trees functions
  - Reorganization of locals within test_quick_select
  - De-dent some large code blocks by changing
      if (key)
      {
        LOTS-OF-CODE;
      }
    into
      if (!key)
        continue;
      LOTS-OF-CODE;
Yuchen Pei
MDEV-37829 Call spider_internal_xa_prepare when XA COMMIT ... ONE PHASE

This allows 1PC XA involving spider tables.
Kristian Nielsen
Binlog-in-engine: Fix inconsistency in stmt/trx cache use

There was a bug when a DML used both the stmt and trx cache but the trx
cache is empty (in this case INSERT IGNORE that inserts nothing but still
allocates from a sequence). The code would put the GTID in the stmt cache
but binlog the trx cache with some stray dangling annotate/tablemap event.
Thus a completely invalid event group resulted.

In general, the logic for selecting stmt or trx cache was complicated and
inconsistent in multiple places around the code. This patch improves by
introducing a single binlog_cache_mngr::engine_cache_data() which holds the
logic and is used in all places.

This patch also fixes a few problems with the GTID event binlogged for user
XA, found during debugging.

Signed-off-by: Kristian Nielsen <[email protected]>
Nikita Malyavin
MDEV-37596 enforce_storage_engine has an effect on child global temporary tables

Ignore the option when creating child global temporary tables.
Nikita Malyavin
MDEV-37668 SIGSEGV on DROP TABLE GTT under LOCK TABLES and different server_id

Fixed as MDEV-37673.

This commit only includes test.
Marko Mäkelä
MDEV-37677: Inconsistent flush_list after InnoDB recovery

In commit bea4adcb5a410659803d4aecc4b384a0e7d67d56 (MDEV-35225)
we inadvertently introduced a race condition. Another thread
may invoke buf_page_t::write_complete() between the time
log_sort_flush_list() inserted the block to the list for sorting,
and the time it would apply the sorted list back to buf_pool.flush_list.
In this case, log_sort_flush_list() would neither add the block to
buf_pool.flush_list nor clear the buf_page_t::oldest_modification_
so that it would correctly indicate whether the block is in the list.

log_sort_flush_list(): Simplify the logic, and always add the entire
sorted list to the buf_pool.flush_list, even if they had been written
back during the time we were copying or sorting.

This fixes an anomaly where a subsequent
buf_pool_t::insert_into_flush_list() would end up incrementing
buf_pool.flush_list.count by one too much.

Thanks to Daniel Black for providing an "rr replay" trace of a
failure.
Nikita Malyavin
MDEV-37771 pseudo_slave_mode=1 must not let pseudo_thread_id=0 assignment

Narrow the range to start from 1, set default to MY_THREAD_ID_MAX
Daniel Black
MDEV-27898 CREATE VIEW AS SELECT FROM JSON_TABLE column requires global privileges

JSON_TABLE is marked as a special "*any_db*" table. Because this special
marking is processed all the way though to get_column_grant where
its processed like its in database called "*any_db*". As this
doesn't exist, only those users with global privileges can create
views on a JSON_TABLE.

Under a Prepared Statement protocol a Create_tmp_table is
used for the JSON_TABLE, but it gets assigned an "" database
name.

We correct this to give it "*any_db*" like the SQL parser
indicating that no database is needed.

To commonly correct the fill_effective_table_privileges
by looking explictly for the "*any_db*", those tables that
have this as the database name get SELECT privileges.

While correcting the database for the JSON_TABLE, lets give
it a "json_table" name rather than "(temporary)" for
greater clarity in warning messages.
Yuchen Pei
MDEV-36325 MDEV-36357 Call limit_mode on spider_db_conn instead of spider_db_result in spider_db_store_result

This prevents segv on NULL result_list->current->result on handler
calls that are not the first. Reasons supporting this change:

- spider_db_result::limit_mode is not called anywhere else. The only
  calls to limit_mode method are on spider_db_conn
- it is very unlikely (impossible?) for the connection to change from
  one backend to another between execution of sql statements and
  storing result:

/* in spider_bg_conn_action: */
                if (dbton_handler->execute_sql(
                  sql_type,
                  conn,
                  result_list->quick_mode,
                  &spider->need_mons[conn->link_idx])
// [... 9 lines elided]
                    if (!(result_list->bgs_error =
                      spider_db_store_result(spider, conn->link_idx,
                        result_list->table)))

  this also means it is very unlikely (impossible?) for the backend
  type (dbton_id) to differ between conn->db_conn and
  result_list->current->result, also considering that
  spider_db_result::dbton_id comes from spider_db_conn::dbton_id:

spider_db_result::spider_db_result(
  SPIDER_DB_CONN *in_db_conn
) : db_conn(in_db_conn), dbton_id(in_db_conn->dbton_id)

Since this was the only call to spider_db_result::limit_mode, we also
remove the method altogether.
Nikita Malyavin
MDEV-37681 SIGSEGV on TRUNCATE GTT after failed RENAME under LOCK TABLE

WHILE RENAME was covered in the reference counting feature,
ALTER TABLE REMANE was missed, as it's turned to be a different code
path.

Exit earlier in mysql_alter_table, if global temporary table is opened
in this connection, or in the other connection. Add timeout argument to
simple_rename_or_index_change, which can be 0 to conform Oracle behavior.
Dave Gosselin
MDEV-36125 [NO_]INDEX_MERGE Hint

Introduces NO_INDEX_MERGE and INDEX_MERGE, which control whether or
not index merge strategies are used during query optimization.  Here
is an example query from the tests:

  SET optimizer_switch='index_merge_intersection=off';
  EXPLAIN SELECT /*+ INDEX_MERGE(t1 f4, f2) */ COUNT(*) FROM t1
  WHERE f4 = 'h' AND f2 = 2;

with the hint in place, the query plan will employ the index_merge
intersect strategy (abbreviated EXPLAIN output):

  type Extra
  index_merge Using intersect(f2,f4); Using where; Using index

The presence of the [NO_]INDEX_MERGE hint overrides the optimizer's
choice of keys during the index merge optimization.  As we see in
the above example, keys f2 and f4 and given and the optimizer will
consider only those keys for this query.

When the hint is given without any particular keys, as in
INDEX_MERGE(table), then all keys are considered.  In this case, the
cheapest index merge among the keys should be used.  When
NO_INDEX_MERGE(table) is given, then index merge is disabled for
that table.

When the hint is given with one or more keys, then only those keys
are considered.  In the case of NO_INDEX_MERGE, those keys are
excluded.  This can lead to no merged indexes at all, because
there may not be sufficient row-ordered read columns available for
consideration.

The index merge strategies of intersection, union, and sort union
cannot themselves be directly controlled via the hints.  In combination
with the optimizer switches for the same, the strategy may be
indirectly controlled but this is not guaranteed.

When the hint directs the optimizer such that insufficient ROR scans
are available, thus leading to a situation where the INDEX_MERGE hint
cannot be honored, the server will emit a warning to that effect.

In the hints module (opt_hints*{cc,h}), this commit adds some
index merge-specific functionality to make interpreting hint state
at callsites in the optimizer cleaner and more intuitive.
Particularly, we add a bit field to the table hints class which
indicates the keys that are marked by an [NO_]INDEX_MERGE hint, if
present.  A new function, index_merge_hint (and associated new
helper functions) relies on this field when interpreting index merge
hint state for the optimizer.

If there are no index merges available prior to attemping to find
a suitable union/sort union, then the optimizer will not attempt
it.  This change results in optimizer trace output which does not
include the 'analyzing_index_merge_union' block when there are no
merges.

Parts of this implementation based on MySQL commit
ebcb981807e3d91a64782e89d48e1a25622eafea
Nikita Malyavin
MDEV-37720 use-after-free on CREATE OR REPLACE GTT under pseudo_slave_mode

... and LOCK TABLES.

Global temporary tables data is not replicated, and pseudo_slave_mode is
used internally for partial replication testing.

Hence forbid opening GTT (i.e. create child GTT handles) under
pseudo_slave_mode, and forbid setting pseudo_slave_mode=1 whenever child
GTT handles are open.
Nikita Malyavin
MDEV-37798 Incomplete savepoint support with global temporary tables

Add empty savepoint support callbacks to global_temporary_tp
Marko Mäkelä
MDEV-37659 CHECK TABLE…EXTENDED error on a column prefix index

trx_undo_prev_version(): Prevent unsafe BLOB access.

Tested by: Saahil Alam
Reviewed by: Thirunarayanan Balathandayuthapani
Yuchen Pei
MDEV-37568 Fix remote table key direction in spider/bugfix.index

Ideally spider should throw an error when local and remote tables have
conflict definitions.
Yuchen Pei
MDEV-28363 remove #ifdef SPIDER_use_LEX_CSTRING_for_Field_blob_constructor