Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Syed Mohammed Nayyar
MDEV-40201: bound the trailing hex read in my_mb_wc_filename

Problem:
my_mb_wc_filename() decodes the my_charset_filename '@HHHH' escape, which
is 5 bytes, but the length guard only required 4 (s[0..3]) before the hex
branch read the 4th hex digit at s[4]. A truncated escape whose end
pointer sits at s+4 (reachable through well_formed_length()/charpos() on a
my_charset_filename string with a tight, non-terminated bound) read one
byte past e.

Fix/Solution:
A 4-byte escape is not valid, so require all 5 bytes up front: s + 5 > e
now returns MY_CS_TOOSMALL5. The read of s[4] keeps the s[3] guard,
because 'e' may be fake (see the strconvert() note above the function):
a NUL-terminated input such as "@00\0" with an over-long end must still
stop at the terminator instead of reading s[4]. Added a strings-t case
covering both the tight-end (MY_CS_TOOSMALL5) and the NUL-terminated
(MY_CS_ILSEQ) truncations.
Daniel Black
tests: pass test regardless of timezone
Daniel Black
build: take server WITH_{UB,A}SAN settings
Georgi (Joro) Kodinov
MDEV-17746: perfschema.dml_threads failed in buildbot with wrong errno

The test was trying to update the performance_schema.threads row for
the current connection. And if that row was not found there's nothing
to update hence no update not allowed error.
If the maximum number of instrumented threads is hit the assumption
that there's always a row in performance_schema.thread for the current
thread does not hold. This can happen under heavy concurrent load.
mtr sets the maximum number of instrumented threads to 400. This is
not much.
Stabilized the test as follows:
* Added a SELECT with the same conditon to ensure there's a row to
  be updated by the UPDATE following the select.
* Increased the performance_schema_max_thread_instances to 10k
    for the test.
Hemant Dangi
MDEV-38843: bump wsrep-lib on 10.11 to the commit-order fix

Issue: the 10.6->10.11 merges dropped the wsrep-lib pin bump, so the
commit-order fix (5eeef4009d5) never reached 10.11.

Solution: advance wsrep-lib to 5eeef4009d5 and restore the
Wsrep_client_service::notify_state_change() override it requires.
Daniel Black
MDEV-40165 10.11 JSON functions don't respect KILL QUERY

Or max_statement_time limit.

Implement interuptability of:
* JSON_OVERLAPS
* JSON_LENGTH
* JSON_DEPTH

As json_report_error now processes the killed there's
no need of thd->check_killed(). Removed this from a few
locations.

Verified that the path after every json_scan_next, the
location of checking the killed_ptr, reports errors
correctly.
Daniel Black
MDEV-40237: Parition info nullptr dereference

thd->lex is null in InnoDB purge thd. The Item_field
can take a NULL context as a pointer for resolving
field names.
Sergei Petrunia
Fix comment: remove mention of non-existent UNCACHEABLE_PREPARE
Daniel Black
MDEV-40175: JSON_VALID don't respect KILL QUERY

Or max_statement_time limit.

Change the json_valid to take a json_engine_t argument.
Adjust the Item_json_valid to have a json_engine_t and
to report an error.

Note this means that invalid json syntax now has a
note on any json error. Raising this to a warning can
cause check constraints of JSON_VALID() to result
in ER_JSON_SYNTAX rather than ER_CONSTRAINT_FAILED
(valided in main.type_json test).

Consequential changes in debug build to json acl
testing validation and unit test having a dummy json_engine_t.

json_normalize also uses the same engine that was
passed as its argument to test json_valid. Because
the character set may have changed, if there's an error
adjust the je->s.c_str pointer. Its only the offset that
is currently used in the error messages anyway so there
may be offset errors depending on the original character
set.
Thirunarayanan Balathandayuthapani
MDEV-27569  Valgrind/MSAN errors in ha_partition::swap_blobs() for BLOB not in secondary index

Problem:
=======
On a partitioned table containing a BLOB/TEXT column, an ordered
index scan calls ha_partition::swap_blobs() for every buffered
row, because it is being called only when table has blob fields.

When the buffered row comes from a secondary index that does not
cover the blob (e.g. KEY(f2) on (f2,f1)), InnoDB's
row_sel_store_mysql_rec() populates only the templated columns.
The blob column's null bit and its value-pointer bytes in
table->record[0] are left uninitialized. The partition handler
memcpy() this record into rec_buf, then swap_blobs() evaluates
blob->is_null(), reading the uninitialized null bit leads to
"use-of-uninitialized-value".

Solution:
=========
row_sel_store_mysql_rec(): When the record is being stored
from a secondary index, seed the NULL bitmap of mysql_rec
from prebuilt->default_rec before populating the templated columns.
Columns present in the index template have their NULL bit rewritten
explicitly while the row is stored, so only the columns absent from
the template (the uncovered blob) rely on this seed.
default_rec marks such nullable columns as NULL and preserves the
reserved NULL bit, so the uncovered blob is now defined as NULL.

Because of this, ha_partition::swap_blobs() skips it via its
"!bitmap_is_set(read_set) || blob->is_null()" guard and
never calls Field_blob::cached()/get_ptr(). This avoids not
only the uninitialized null-bit read and the follow-on
uninitialized blob value-pointer read.
Dave Gosselin
MDEV-33524:  marked_for_read assertion with virtual column

A couple of things interact in this scenario.

1. A virtual column whose value depends on session state, such as a
VCOL calling DATE_FORMAT, has fix_fields (via fix_expr) run on it
again before each INSERT.  For a table reused from the cache this runs in
Vcol_expr_context before setup_tables sets the table map.  The test
first opens t1 with a SELECT that fails before setup_tables, leaving
it cached with map zero.

2. When the connection character set needs conversion, CONCAT (more
generally, any charset aggregation) wraps that column in a charset
converter.  The converter evaluates any argument reported as
constant, so it calls val_str() on the column as part of
fix_fields, before a row exists and without the column in the read
set.  On DEBUG builds this fails the marked_for_read assertion;
otherwise it reads an unread column.

Consequently:
When a table instance is opened, fix_fields runs with table->map
already nonzero, but the fix_fields (via fix_expr) call from vcol_fix_expr
did not have it nonzero; so set table->map=1 during Vcol_expr_context::init.
The original Vcol_expr_context::init() (MDEV-24176) called
init_lex_with_single_table, which set table->map= 1 as a side effect.
A later merge of 10.2 into 10.3 (6f6c74b0d18) replaced that call with
a backup of table->expr_arena, dropping the side effect.
The destructor of the Vcol_expr_context will reset the map back to its
original value.
Dave Gosselin
MDEV-33524:  marked_for_read assertion with virtual column

vcol.vcol_misc deadlocks under --view-protocol at the
FLUSH TABLES WITH READ LOCK block.  The main connection holds
the global read lock.  The SELECT COUNT(*) FROM t1 query becomes
CREATE OR REPLACE VIEW on a separate connection, and that DDL
blocks on the read lock and never returns.  So the same mysqltest
client never reaches unlock tables.  The fix is to wrap only that
one SELECT in the view_protocol guards.
Daniel Black
MDEV-28404 json killed_ptr cleanup

Since 24ee5fd6bf6065ea693287e5705f3e26bb85f9fc the result of the killed_ptr
is part of the je->s.error after json_scan_next.
Georgi (Joro) Kodinov
MDEV-17746: perfschema.dml_threads failed in buildbot with wrong errno

The test was trying to update the performance_schema.threads row for
the current connection. And if that row was not found there's nothing
to update hence no update not allowed error.
If the maximum number of instrumented threads is hit the assumption
that there's always a row in performance_schema.thread for the current
thread does not hold. This can happen under heavy concurrent load.
mtr sets the maximum number of instrumented threads to 400. This is
not much.
Stabilized the test as follows:
* Added a SELECT with the same conditon to ensure there's a row to
  be updated by the UPDATE following the select.
* Increased the performance_schema_max_thread_instances to 50k
    for the test.
Daniel Black
MDEV-40165: json_normalize (fix)

14c16e02b261ad8e41ea6f80ebd4f42403793aff changed the
interface for json_normalize to take a json_engine_t so that
there could be some error checking.

Unfortunately ColumnStore depended on this interface.

Restored original interface of json_normalize() and
added a json_normalize_engine() that take as engine as the
argument.
Daniel Black
MDEV-31554 Cursor protocol cuts off json boolean

In Item_func_json_array_append::fix_length_and_dec,
Item_bool::max_char_length() was being used to
calculate the length that is required to fit a boolean.

Previously it was returning 1, which doesn't fit
"false".

Change Item_bool to return strlen("false"), 5.
Daniel Black
MDEV-37869 binlog.binlog_unsafe fails on Windows

And macos.

The order of the Table_map events don't actually matter in this test.
The important aspect is the Annotate_rows and Write_rows_v1 events.

Lets exclude the Table_map events in the show_binlog_events.inc
aspect of this test.

The show_events.inc has been extended to support this
$skip_tablemap_events=1 variable. $skip_checkpoint_event only has
two usages so we haven't tried to support both together until
needed.
Jaeheon Shim
MDEV-39932 Accept aggregated outer columns in subquery

Under ONLY_FULL_GROUP_BY, a query that aggregates an outer column inside
a subquery is wrongly rejected. This is fixed in
Item_field::fix_outer_field by not appending the field to
select->join->non_agg_fields when thd->lex->in_sum_func is not null.

Furthermore, in Item_sum::check_sum_func, for all outer fields that are
not aggregated at their SELECT_LEX's nest level, we append these fields
to sel->join->non_agg_fields in order to ensure that
ER_WRONG_FIELD_WITH_GROUP is still raised for invalid aggregation.
forkfun
MDEV-23444 ASAN dynamic-stack-buffer-overflow or Assertion `precision > 0'
failed in decimal_bin_size with div_precision_increment=0

A signed numeric value of display length 1 (WEEKDAY(), DAYOFWEEK(), @v:=<int>)
had decimal_precision() == 0: my_decimal_length_to_precision() subtracted a
digit for the sign with no lower bound. Turning such a value into a DECIMAL
(division, AVG(), a UNION column) produced precision 0, which tripped the
`precision > 0' assertion in decimal_bin_size() on store, GROUP BY or filesort.

Clamp my_decimal_length_to_precision() to a minimum of 1 so precision is never 0.

Side effect: in a query CREATE TABLE t1 SELECT * FROM (SELECT 1 as a,(SELECT a)) a;
`(SELECT a)` column now reports width 2 (digit + sign).
It's an invalid query that's supported for historical reasons, and all valid queries
did not change their results
Daniel Black
json_key_value - simplify error handling

Same result, just consolidating the implementation.
bsrikanth-mariadb
MDEV-40006: Prepared Statement Crash in varchar_upper_cmp_transformer() for '?'

Item_func_in::varchar_upper_cmp_transformer() clones Item_func_in
and its arguments and doesn't check if cloning has succeed.

Item_param (PS parameter, '?') doesn't implement cloning so will return
NULL from clone(). This will cause a crash.

Fixed by making varchar_upper_cmp_transformer() check clone results for NULL.
(Adding cloning support to Item_param is out of scope of this patch).
Georgi (Joro) Kodinov
MDEV-17746: perfschema.dml_threads failed in buildbot with wrong errno

The test was trying to update the performance_schema.threads row for
the current connection. And if that row was not found there's nothing
to update hence no update not allowed error.
If the maximum number of instrumented threads is hit the assumption
that there's always a row in performance_schema.thread for the current
thread does not hold. This can happen under heavy concurrent load.
mtr sets the maximum number of instrumented threads to 400. This is
not much.
Stabilized the test as follows:
* Added a SELECT with the same conditon to ensure there's a row to
  be updated by the UPDATE following the select.
* Increased the performance_schema_max_thread_instances to 10k
    for the test.
Daniel Black
MDEV-39742 11.4 JSON functions not interuptable

The JSON functions added in 11.4 where not interuptable
with a KILL QUERY or exceeding the max_statement_time.

* JSON_ARRAY_INTERSECT
* JSON_OBJECT_TO_ARRAY
* JSON_SCHEMA_VALID

This behaviour is corrected by setting the killed_ptr
of the json_engine_t structure.

JSON_KEY_VALUE was added in 11.4, however as it doesn't call
json_next_value(), its processing doesn't check the killed pointer
of the json engine. So its quick anyway.

JSON_OBJECT_FILTER_KEYS, has some structural problem. These
will be addressed in MDEV-39941 and the KILL QUERY/max_statement_time
will be implemented then.

Reviewed by: Rucha Deodhar
Abhishek Bansal
MDEV-38033: JSON_SCHEMA_VALID returns wrong result for array of objects

(11.4 backport from 12.3)

Fix a bug in Json_schema_items::validate where it was missing a
json_skip_level() call. Without this skip, the validator would
incorrectly recurse into non-scalar array elements (like objects)
and try to validate their internal keys/values against the array's
item schema.
Vladislav Vaintroub
MDEV-37781 ASAN build crashes on deep query with low thread stack

check_stack_overrun() was compiled out under ASAN, so a deeply nested
expression recursed in Item_func::fix_fields() until the stack was
exhausted and the server crashed instead of reporting
ER_STACK_OVERRUN_NEED_MORE.

Since MDEV-34533 (Monty) the stack usage seems to be accounted correctly
under ASAN via my_get_stack_pointer(), so the check seems to works there
too.

Fix:
Remove the #ifndef __SANITIZE_ADDRESS__  guard from check_stack_overrun()

Add a test for ER_STACK_OVERRUN_NEED_MORE
Daniel Black
MDEV-29542: enable --view-protocol on set_password test

By using the case correct User,Host,Password from the
mysql.user view.
Daniel Black
MDEV-40313 mariadb client creates `.mariadb_histor` instead of `.mariadb_history`

`histfile_size` in `client/mysql.cc` is calculated using the string
"/.mysql_history" (15 chars) rather than "/.mariadb_history" (17 chars).

Because the buffer is only allocated to that shorter length, the resulting
filename is truncated by 2 characters.

Thanks David Lu for the bug report and diagnosis.
Daniel Black
Merge 10.11 into 11.4
Kristian Nielsen
MDEV-40057: Virtual column included in after-image with unique BLOB index, asserts

Do not put any virtual column values into the before-image or after-image of
row events.

These virtual column values were causing assertion and probably other issues
as well. Concretely, a UNIQUE index on (int_col, blob_col) where the int_col
is being updated would include the hidden virtual column for the index in
the after-image, without including the value of the blob_col in
binlog_row_image=MINIMAL. This would cause an assertion during
TABLE::update_virtual_fields() due to missing bit in the bitmap.

Even if it doesn't cause an assertion, including the virtual column values
makes no sense, as they are recomputed anyway on the slave when applying the
row events.

Reviewed-by: Monty <[email protected]>
Brandon Nesterenko <[email protected]>
Signed-off-by: Kristian Nielsen <[email protected]>
Daniel Black
MDEV-40165: JSON_EQUAL/JSON_NORMALIZE dont respect KILL QUERY

Or max_statement_time limit.

JSON_EQUAL/JSON_NORMALIZE also didn't report warnings
on invalid JSON inputs.

Change the json_normalize function to take a json_engine_t
as a argument and implement the sql/item_jsonfunc.cc calls
to have a json_engine_t.

To the JSON_EQUALS and JSON_NORMALIZE, add error handling
to produce an error when there was one. compare_nested_object,
part of JSON_OVERLAPS, uses a new json_engine and
copies the error/position to the returning function.

json_normalize expects the incoming je structure to have
a valid, or nullptr, killed_ptr, as json_start will reset this.
Vladislav Vaintroub
MDEV-34074 server_audit crashes in get_loc_info() with NULL loc_info

THDVAR(thd, loc_info) can return NULL if the plugin was previously
uninstalled. get_loc_info() and its callers dereferenced it.

Guard against a NULL loc_info in get_loc_info(), auditing() and
log_current_query(). Such a session is not audited.
Daniel Black
MDEV-40175: JSON_VALID (postfix)

a90779098b8683bcf87201e21f1c6d21abdbb56d changed the
interface to the json_valid to take a json_engine_t
so that it could be interupted and so that warnings
could become visible.

On merge from that 10.11 to 11.4 it was discovered
that the ColumnStore engine uses this interface.

Restored the original json_valid function exactly
how ColumnStore expects this and added renamed
the function with json_engine_t to be json_valid_engine.

This follows the same convention as in the commit
756fc6fd9ae49a1310f2dac6febf83cb75c6f557 that was
for json_normalize, that ColumnStore also used.
Thirunarayanan Balathandayuthapani
MDEV-40332 InnoDB: Defragmentation of BASE_IDX in SYS_VIRTUAL failed: Data structure corruption

Problem:
========
- While shrinking the system tablespace, InnoDB defragments the
system tables to move used pages out of the extents
near the end of the file so the tail can be truncated.
defragment_level() relocates every page of a to-be-moved extent
and, to do so, rewrites the node pointer in the page's parent.

A B-tree root page has no parent node pointer, so it is
never recorded in m_parent_pages.
When the root of a system-table index happens to reside in an extent
that was selected for relocation, the parent lookup fails and
defragmentation is aborted with DB_CORRUPTION even though nothing is
corrupt.

Solution:
=========
  Because a root page cannot move, the system tablespace cannot
shrink below the highest root page, and relocating any
page at or below it cannot reduce the file size.
Exclude that region from relocation up front.

SpaceDefragmenter::max_root_extent(): New function returning the
highest extent that holds a root page of a system-table index

SpaceDefragmenter::find_new_extents(): Use max_root_extent()
together with the minimum tablespace size as the lower bound
(floor) of the relocation scan, so no extent at or below the
highest root is ever added to the relocation map.
This avoids the false corruption and also avoids reserving a
destination extent for a move that would never happen.
The free tail above the highest root is still reclaimed by truncation.
Kristian Nielsen
Fix excessive allocations of memory for table object bitmaps

Reviewed-by: Monty <[email protected]>
Signed-off-by: Kristian Nielsen <[email protected]>
Teemu Ollakka
MDEV-40222 Prevent MTR hang when waiting for wsrep_ready

A query against a server that is up but wedged can connect yet never
return, so the loop-count bound in wait_wsrep_ready() did not actually
limit the wait and MTR could hang until the suite timeout fired.

Add an optional $timeout to run_query_output(): the mysql client is
now spawned via My::SafeProcess->new and waited for with
wait_one($timeout), killing the client and returning non-zero if it
does not finish in time.

Bound wait_wsrep_ready() by a wall-clock deadline (start_timer)
instead of a loop count, and pass the remaining time to each query so
no single hung client can exceed the overall server startup budget.
Hemant Dangi
MDEV-40280: FreeBSD galera.galera_max_ws_rows test failure

Issue:
For unknown error numbers, BSD-derived libc (macOS, FreeBSD, ...) formats
"Unknown error: <n>" with a colon, while glibc uses "Unknown error <n>".
my_strerror()'s colon-stripping (MDEV-35578) was gated on __APPLE__ only,
so FreeBSD output diverged from the .result.

Solution:
Normalize by behavior instead of by platform: always strip the colon after
"Unknown error". glibc never emits it, so it is a no-op there, and
macOS/FreeBSD/other BSD libc are all covered without enumeration. This is
what MDEV-35578 already intended ("consistent across the platforms ...
when present"); the __APPLE__ guard was just narrower than that intent.
Runs only in the unknown-error path.
Daniel Black
merge - json_normalize changes
Georgi (Joro) Kodinov
MDEV-17746: perfschema.dml_threads failed in buildbot with wrong errno

The test was trying to update the performance_schema.threads row for
the current connection. And if that row was not found there's nothing
to update hence no update not allowed error.
If the maximum number of instrumented threads is hit the assumption
that there's always a row in performance_schema.thread for the current
thread does not hold. This can happen under heavy concurrent load.
mtr sets the maximum number of instrumented threads to 400. This is
not much.
Stabilized the test as follows:
* Added a SELECT with the same conditon to ensure there's a row to
  be updated by the UPDATE following the select.
* Increased the performance_schema_max_thread_instances to 50k
    for the test.