Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Dave Gosselin
MDEV-38136: Prevent elimination of tables in a FULL OUTER JOIN

Prevent elimination of tables participating in a FULL OUTER JOIN during
eliminate_tables as part of phase one FULL OUTER JOIN development.

Move the functionality gate for FULL JOIN further into the codebase: convert
LEX::has_full_outer_join to a counter so we can see how many FULL JOINs
remain which makes the gate work correctly after simplify_joins and
eliminate_tables are called.

Fixes an old bug where, when running the server as a debug build and in
debug mode, a null pointer deference in
Dep_analysis_context::dbug_print_deps would cause a crash.
Dave Gosselin
MDEV-38508: Constant table detection

If a table that's in a FULL OUTER JOIN is found to be a const
table, then don't allow the constant table optimization to
take place.

Later, when we support FULL OUTER JOIN on the inner side of
other join types then we may be able to relax this restriction.
ParadoxV5
MDEV-39788 CHANGE MASTER savefiles read and write one line to many

The MDEV-37530 refactor missed the detail that the line counts in
`@@master_info_file` and `@@relay_log_info_file`
also counted the line-count line itself.
Consequently, although fresh 12.3 installations
and downgrades to earlier versions are unaffected,
upgrades from pre-12.3 will encounter surprises when
the 12.3 upgrade tries to read and discard the extra line:
* `master_use_gtid` is reset to DEFAULT.
* `@@relay_log_info_file` fails to load
  unless its data has previously shrunk.

Since downgrading remained compatible,
this commit fixes this content difference by correcting
the code to read and write one fewer (unused) line.

Reviewed-by: Brandon Nesterenko <[email protected]>
Vladislav Vaintroub
x
ParadoxV5
Merge branch 'MDEV-39788.test' into MDEV-39788
ParadoxV5
whoops. (RIP our CI today)
Dave Gosselin
MDEV-38692: COALESCE() on NATURAL FULL JOIN result sets

FULL JOIN yields result sets with columns from both tables participating in
the join (for the sake of explanation, assume base tables).  However,
NATURAL FULL JOIN should show unique columns in the output.

Given the following query:
  SELECT * FROM t1 NATURAL JOIN t2;
transform it into:
  SELECT COALESCE(t1.f_1, t2.f_1), ..., COALESCE(t1.f_n, t2.f_n) FROM
    t1 NATURAL JOIN t2;

This change applies only in the case of NATURAL FULL JOIN.  Otherwise,
NATURAL JOINs work as they have in the past, which is using columns
from the left table for the resulting column set.
ParadoxV5
fixup

use RESET SLAVE ALL in case other tests restart the server without
specifying `--skip-slave-start` (why not default in MTR I do not know)
Vladislav Vaintroub
MDEV-14443 - address some review comments

Make sure deny_entry_to_json does not fail in memory allocation,
by providing StringBuffer with a large enough statically allocated
size DENY_JSON_ENTRY_BUFSIZE, currently 1230 bytes.

update_deny_entry() - handle json.append() errors
Dave Gosselin
MDEV-37932: Parser support FULL OUTER JOIN syntax

Syntax support for FULL JOIN, FULL OUTER JOIN, NATURAL FULL JOIN, and
NATURAL FULL OUTER JOIN in the parser.

While we accept full join syntax, such joins are not yet supported.
Queries specifying any of the above joins will fail with
ER_NOT_SUPPORTED_YET.
Marko Mäkelä
WIP MDEV-39092, and back up non-InnoDB files
Dave Gosselin
MDEV-39746: FULL JOIN with a nested join on the right loses rows

The outermost FULL JOIN's right operand can be a nested join rather
than a single base table.  The parser places the nest on the right
when the outermost FULL JOIN's ON is the last one written, because the
parser keeps the outermost FULL JOIN pending until its ON arrives, and
the inner FULL JOINs reduce first into a nest that becomes the right
operand.
alloc_full_join_duplicate_filters allocates the fj_dups filter on a
JOIN_TAB carrying JOIN_TYPE_FULL | JOIN_TYPE_RIGHT, so with the
FULL|RIGHT bits on the nest, which is never a JOIN_TAB, no filter was
allocated and the null complement pass never fired.  The unmatched
rows from the right side were never emitted, producing a result with
missing rows.

Add swap_full_join_sides, called from rewrite_full_outer_joins
when a FULL JOIN survives simplify_joins with a leaf on the left
and a nested join on the right.  FULL JOIN is symmetric on its
operands, so swapping does not change query semantics; after the
swap the leaf carries the FULL|RIGHT bits and the rescan target
is a single base table.
ParadoxV5
Test for MDEV-39788

MDEV-39788 found that the recent refactor on the `main` (now 12.3)
branch forgot that the line count includes the line count line itself.
This test checks future changes in any supported
version that they don’t make this mistake again.
Vladislav Vaintroub
x
Dave Gosselin
Reject a nested join on the right of a rewritten FULL JOIN

check_full_join_base_tables runs before simplify_joins and rejects the
disallowed FULL JOIN shapes that are visible in the parse tree.
simplify_joins can rewrite a FULL JOIN to a LEFT, RIGHT, or INNER
JOIN, so sometimes disallowed queries appear only afterward.

Add check_full_join_after_simplify, called from optimize_inner once
simplify_joins is done, to reject unsupported queries after
rewrite by simplify_joins.
Dave Gosselin
MDEV-37933: Rewrite [NATURAL] FULL OUTER to LEFT, RIGHT, or INNER JOIN

Rewrite FULL OUTER JOIN queries as either LEFT, RIGHT, or INNER JOIN
by checking if and how the WHERE clause rejects nulls.

For example, the following two queries are equivalent because the
WHERE condition rejects nulls from the left table and allows matches
in the right table (or NULL from the right table) for the remaining
rows:

  SELECT * FROM t1 FULL JOIN t2 ON t1.v = t2.v WHERE t1.v IS NOT NULL;
  SELECT * FROM t1 LEFT JOIN t2 ON t1.v = t2.v;

  SELECT * FROM t1 FULL JOIN t2 ON t1.v = t2.v WHERE t1.a=t2.a;
  SELECT * FROM t1 INNER JOIN t2 ON t1.v = t2.v WHERE t1.a=t2.a;
Jan Lindström
MDEV-39783 : Galera test failure on MDEV-35018

Test did echo wsrep_apply_waits+1 value, but this value
is dependent what was value of the wsrep_apply_waits when
test started. If previous test run by same worker
increased the value of wsrep_apply_waits, expected
value 1 in result file would be different than current
value. Removed output as it does not give any real value,
because test already has wait_condition for expected
increase of wsrep_apply_waits value.
Dave Gosselin
MDEV-39569: Skip FULL JOIN rewrite to inner side of an outer join

Prevent simplify_joins from rewriting a chained FULL JOIN into a query
where a FULL JOIN could end up on the inner side of another outer
join.  Of course, this means that we will have a null complement pass
that the rewritten query would have avoided.  Once we support FULL
JOINs on the inner side of outer joins, in phase 3, then we can relax
this constraint.
ParadoxV5
Merge branch 'MDEV-39788.test' into MDEV-39788
Marko Mäkelä
WIP MDEV-14992 BACKUP SERVER

This introduces a basic driver Sql_cmd_backup, storage engine interfaces,
and basic copying of InnoDB data files.
On Windows, we pass a target directory name; elsewhere, we pass a
target directory handle.

backup_target: A structured data type to represent a directory or a
stream. On Microsoft Windows, we must use directory paths because
there is no variant of CopyFileEx() that would work on file handles.

copy_entire_file(): A file copying service for POSIX systems.

copy_file(): A sparse file-copying service for POSIX systems.

backup_context: An InnoDB backup context, attached to trx->lock.backup
so that context can exist between InnoDB_backup::end(), which is
releasing all locks, and InnoDB_backup::fini() in the same thread,
which is expected to finalize the backup without modifying files
in the server data directory.

fil_space_t::write_or_backup: Keep track of in-flight page writes and
pending backup operation. We must not allow them concurrently, because
that could lead into torn pages in the backup.

fil_space_t::backup_end: The first page number that is not being backed up
(by default 0, to indicate that no backup is in progress).

TRX_STATE_BACKUP: A special InnoDB transaction state indicating association
with BACKUP SERVER, which allows us to pass some context in trx_t from
innodb_backup_end() to innodb_backup_finalize().

log_t::backup: Whether BACKUP SERVER is in progress. The purpose of this
is to make BACKUP SERVER prevent the concurrent execution of
SET GLOBAL innodb_log_archive=OFF or SET GLOBAL innodb_log_file_size
when innodb_log_archive=OFF.

log_sys.archived_checkpoint: Keep track of the earliest available
checkpoint, corresponding to log_sys.archived_lsn. This reflects
SET GLOBAL innodb_log_recovery_start (which is settable now), for
incremental backup.

buf_flush_list_space(): Check for concurrent backup before writing each
page. This is inefficient, but this function may be invoked from multiple
threads concurrently, and it cannot be changed easily, especially for
fil_crypt_thread().
ParadoxV5
ye smol problem with test-result pairs is that it’s easy to forget to update the result
ParadoxV5
Merge branch 'MDEV-39788.test' into MDEV-39788
forkfun
MDEV-24557 mariadb-dump: translate MySQL 8.x user/grant syntax for MariaDB import

  mariadb-dump --system=users (and --system=all) can now capture a
  MySQL 8.0+ source and emit the users, roles and grants as statements
  that load on MariaDB. The dump targets MariaDB only (migration is
  one-way), so each statement is emitted once, in MariaDB form.

  - privileges: mapped to a MariaDB equivalent where one exists
    (e.g. BINLOG_ADMIN -> BINLOG REPLAY, BINLOG ADMIN), otherwise dropped
    with a # WARNING;
  - CREATE USER clauses, that MariaDB has no equivalent for (PASSWORD HISTORY,
    PASSWORD REUSE INTERVAL, DEFAULT ROLE, ...): dropped;
  - auth plugins, that MariaDB does not ship (e.g. sha256_password): the
    IDENTIFIED clause is dropped and the account is forced ACCOUNT LOCK;
  - roles and the role-admin hierarchy, default roles, and partial
    REVOKE (no MariaDB equivalent -> # WARNING).
Dave Gosselin
Update table_elim for FULL JOIN base table check

Two EXPLAIN queries in table_elim place a nested join on the right
side of a FULL JOIN.  Phase 2 supports only base tables there, so
check_full_join_base_tables rejects them with
ER_FULL_JOIN_BASE_TABLES_ONLY.
Sergei Petrunia
[No MDEV#] Invalid conversion is done when reading JSON

When reading JSON strings. we call json_unescape_to_string()
with a StringBuffer with "binary" charset.

That causes conversion from utf8mb4 (data in JSON is in this
charset) to "wide characters".

Create the StringBuffer with utf8mb4 to make sure no conversion
is done.

Now with unit test.
Dave Gosselin
MDEV-37995: FULL OUTER JOIN name resolution

Allow FULL OUTER JOIN queries to proceed through name resolution.

Permits limited EXPLAIN EXTENDED support so tests can prove that the
JOIN_TYPE_* table markings are reflected when the query is echoed back by the
server.  This happens in at least two places:  via a Warning message during
EXPLAIN EXTENDED and during VIEW .frm file creation.

While the query plan output is mostly meaningless at this point, this
limited EXPLAIN support improves the SELECT_LEX print function for the new
JOIN types.

TODO: fix PS protocol before end of FULL OUTER JOIN development
Dave Gosselin
MDEV-39014: FULL JOIN Phase 2

In phase 1, FULL [OUTER] JOIN was only supported when simplify_joins()
could rewrite it into an equivalent LEFT, RIGHT, or INNER JOIN based
on NULL-rejecting WHERE predicates.  Queries that could not be
rewritten raised ER_NOT_SUPPORTED_YET.  (Phase 1 was not released.)

This commit removes that restriction by adding proper support for FULL
JOIN by executing a 'LEFT JOIN pass' that emits matched rows and left
null-complemented rows, then a second "null-complement" pass which
rescans the right table to emit null-complement rows that were never
matched.

FULL JOIN supports nested joins on the left of the FULL JOIN,
NATURAL FULL JOIN, semi-joins, CTEs / derived tables (kept
materialized when they participate in a FULL JOIN), prepared
statements, stored procedures, and aggregates.  Examples:

  SELECT * FROM (d1 FULL JOIN d2 ON d1.a = d2.a)
              FULL JOIN t3 ON d1.a = t3.a;

  SELECT * FROM t1 NATURAL FULL JOIN t2;

  SELECT * FROM t1 INNER JOIN t2 FULL JOIN t3 ON t1.a = t3.a;

  PREPARE st FROM
    'SELECT COUNT(*) FROM t1 FULL JOIN t2 ON t1.a = t2.a';

Limitations:
  - Statistics and cost estimates for the null-complement pass have
    not been fully implemented; the optimizer may under- or
    over-estimate FULL JOIN costs in plans involving multiple
    FULL JOINs.  Again, a follow-up will optimize the cost calculations.
  - Optimizations for constant tables not fully supported.
  - Nested tables on the right side of a FULL JOIN are not yet supported.
Dave Gosselin
MDEV-38502: FULL OUTER JOIN get correct sargable condition

Fetches the ON condition from the FULL OUTER JOIN as the sargable condition.
We ignore the WHERE clause here because we don't want accidental conversions
from FULL JOIN to INNER JOIN during, for example, range analysis, as that
would produce wrong results.

GCOV shows that existing FULL OUTER JOIN tests exercise this new codepath.
ParadoxV5
Merge branch 'MDEV-39788.test' into MDEV-39788
Dave Gosselin
MDEV-38502: FULL OUTER JOIN get correct sargable condition

Move the temporary gate against FULL OUTER JOIN deeper into the
codebase, which causes the FULL OUTER JOIN query plans to have
more relevant information (hence the change).  In some cases, the
join order of nested INNER JOINs within the FULL OUTER JOIN changed.

Small cleanups in get_sargable_cond ahead of the feature work in
the next commit.
Alexander Barkov
MDEV-39518 Allow prepared statements in stored functions in assignment right hand

In progress
Dave Gosselin
Remove unnecessary and unused 'top' parameter from simplify_joins.
ParadoxV5
didn’t notice that null lists were allowed in pre-12.3
Sergei Petrunia
Update test results
Daniel Bartholomew
bump the VERSION