Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Oleksandr Byelkin
Merge branch '10.6' into bb-10.11-release
Lawrin Novitsky
ODBC-502 Driver can read and send to server memore past query end

This happened if UTF16 character with UnixODBC(i.e. Linux and not
Window) would have lower byte as the high byte in a surrogate pair
leading unit. One of helper functions erroneously took such character as
leading unit of a surrogate pair, and vise versa - would not recognize
real leading unit as such. That could lead to calculated byte length
would be longer, than the real one, it will be used for query text
transcoding to connection charset and thus memory past he end of query
read and sent to the server.
SQLNativeSqlW had the problem caused be similar reasons, but it did not
need length in char calculation, but in SQLWCHAR units.
The tast has been added and SQLNativeSqlW test has been extended.
  • codbc-benchmark: 'dojob rm -rf win64 && mkdir win64 && cd win64 && cmake ../src -G "Visual Studio 17 2022" -DCONC_WITH_MSI=OFF -DCONC_WITH_UNIT_TESTS=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo -DINSTALL_PLUGINDIR=plugin -DALL_PLUGINS_STATIC=ON && cmake --build . --config RelWithDebInfo || cmake --build . --config RelWithDebInfo' failed -  stdio
bsrikanth-mariadb
MDEV-40384: innodb_gis.geometry fails on replay

The test had innodb_strict_mode turned OFF, when running the test. But,
in the replay, it was enabled, which caused the creation of tables with
KEY_BLOCK_SIZE=16 fail.

Solution is to record the innodb_strict_mode variable in the context, so
that it gets used during the replay.
Sergei Petrunia
Remove incorrectly added sql/opt_sum.cc.orig
forkfun
MDEV-39846 UNIQUE BLOB contains duplicate records after ALTER IGNORE with READ COMMITTED

Force REPEATABLE READ isolation for the duration of mysql_alter_table,
so HA_CHECK_UNIQUE_AFTER_WRITE never triggers during ALTER's exclusive table copy.
(cherry-pick of 46ee272a10d7 - online ALTER TABLE (MDEV-32100))
Oleksandr Byelkin
Merge branch '10.11' into bb-10.11-release
bsrikanth-mariadb
MDEV-40390: compat/oracle.sp-package fails on replay

UDFs are not yet supported in replay mode.

So, disabling test compat/oracle.sp-package
bsrikanth-mariadb
Do not dump stats and const rows for read only engines' tables.

Stats for tables from engines such as Archive, S3, PerfSchema, and
Sequence shouldn't be recorded in the context. Similarly, const row
records should also not be stored in the context.

Added few tests for Sequence's engine tables like seq_1_to_5.
Daniel Black
MDEV-40572 SIGSEGV in JSON_OVERLAPS()

on a truncated nested JSON object with different character sets/collations.

The value->s.c_str and js->s.c_str became invalid once the
function returned which caused the crash.
ARaveala
MDEV-39762: Slave Overflow on Malformed Query_compressed_log_event

A replica can crash when a compressed event from its master carries a
corrupted uncompressed length. A compressed Query event whose length
is near 4GB overwrites the IO thread's stack with the event's content.
A compressed rows event of the same shape makes the replica ask for an
allocation of 4GB. A debug build that gets that memory then fails an
assertion in binlog_buf_uncompress(). On a build where a ulong is 32
bits wide, both events overwrite the IO thread's stack.

The IO thread uncompresses a compressed event before writing the event
to the relay log, and offers the uncompress function 4096 bytes of its
own stack to hold the result. The function computes how large the event
will be once uncompressed, and that size decides where the uncompressed
event goes. An event that fits in the stack buffer is uncompressed
there. A larger event is uncompressed into an allocation the function
makes at that size. With either buffer, the function tells zlib that
the room available is the uncompressed length the event declared.

query_event_uncompress() never bounded the length read out of the
event. The function added the header length to that value, rounded the
sum up for alignment, and cast the result to uint32. The cast dropped
the high bits of a sum above 4GB, so a length near 4GB produced a size
of a few dozen bytes. That size fit the stack buffer, so content of any
size went onto the IO thread's stack. row_log_event_uncompress()
computes that sum in a ulong, which wraps where a ulong is 32 bits
wide. The Query_compressed_log_event constructor and
Rows_log_event::uncompress_buf() size their allocations from the same
unbounded length.

Reject an uncompressed length above MAX_MAX_ALLOWED_PACKET at every
point the length is read, before any size is computed from it. No
master writes a larger length, because max_allowed_packet is capped at
1GB. query_event_uncompress() also keeps that size in a size_t, so the
allocation is made from the whole computed value. Both functions return
the error code for the IO thread to report, so a length over the limit
stops the IO thread with ER_TOO_BIG_FOR_UNCOMPRESS rather than the
generic uncompress error. A compressed event with a corrupted
uncompressed length now stops the replica's IO thread with an error
instead of crashing the replica.

Alexandra Raveala wrote the original patch and its first regression
test, contributed through PR 5413. Brandon Nesterenko saw the work
through for the 10.6 release, replacing the test with one that makes
the master write the corrupt length and reworking how the IO thread
reports the error.

Co-authored-by: Brandon Nesterenko <[email protected]>
Reviewed-by: Kristian Nielsen <[email protected]>
forkfun
MDEV-40584 ST_CROSSES always returns 0 for different-dimension geometries

MDEV-36058 added a dimension-equality check ("Both geometries must
have the same number of dimensions") for SP_OVERLAPS_FUNC, but a
stray fall-through from SP_CROSSES_FUNC into that same case made
CROSSES share it too. CROSSES is defined precisely for geometries
of different dimensions, so any such pair now hit
"if (g1_dim != g2_dim) DBUG_RETURN(0)" and always returned 0.

Give SP_CROSSES_FUNC its own case again, calling
handle_sp_crosses_func_case() directly without the dimension check.
SP_OVERLAPS_FUNC keeps the check.
Sergei Petrunia
MDEV-38701: Optimizer Context Replay: merge into 13.1 tree

Optimizer Context Replay feature allows one to record and replay
a query's Optimizer Context. Optimizer Context includes everything
that one needs to replicate how the Query Optimizer processed the query.

It can be replayed on another to host to debug how the Query Optimizer
processed the query, run what-if scenarios, etc.

== Example recording ==

  set optimizer_record_context=1;
  < Run the query of interest. Typically it's EXPLAIN ...>;
  select context [into dumpfile '/tmp/context.sql']
  from information_schema.optimizer_context;

== Example replay ==
  -- On another machine, just source the script
  source context.sql

This will
* Set relevant system variables to match the recording side;
* Create the dataases, tables and views the query needs;
* Load EITS statistics for the tables;
* Provide the optimizer with other context data
* Finally re-run the query. If it was an EXPLAIN, one should get the
  same output as on the recording side.
forkfun
MDEV-38472 Assertion in Diagnostics_area::set_error_status on UPDATE...LIMIT DEFAULT

Binding DEFAULT to a LIMIT ? placeholder via EXECUTE ... USING is never
rejected at bind time. unit->set_limit() later evaluates it and sets
ER_INVALID_DEFAULT_PARAM in the Diagnostics_area, but SQLCOM_UPDATE (and
SQLCOM_UPDATE_MULTI, SQLCOM_INSERT_SELECT/REPLACE_SELECT) proceeded into
open_tables() regardless. A concurrent metadata change (e.g. ALTER TABLE
FORCE) then triggers Reprepare_observer::report_error(), which tries to
set ER_NEED_REPREPARE on an already-set DA and hits the assert.

Add the same thd->is_error() check after unit->set_limit() (as in
SQLCOM_DELETE), aborting before open_tables() is reached.
Brandon Nesterenko
MDEV-40644: Slave SQL Thread Overflow on Malformed Table_map_log_event

A slave can crash when its master sends a Table_map event whose table
name is longer than an identifier can be. The SQL thread applying the
event copies the name into a fixed buffer and overruns it, and the SQL
thread crashes.

A Table_map event carries its database and table names each behind a
one-byte length. The constructor that parses the event sizes a buffer
for each name from its declared length, and do_apply_event() copies
both names out of the event into NAME_LEN+1 byte buffers with strmov().
strmov() copies up to the terminating null. A real identifier is at
most NAME_LEN bytes, so the name fits the buffer and carries that null.

The constructor never bounded the two lengths. Each length is read as a
byte, so each can reach 255. The constructor copied each name with
strncpy() over the declared length, which writes no terminating null
when the source holds none. do_apply_event() then copied the name into
its NAME_LEN+1 byte buffer with strmov(), which ran past the buffer
until it reached a null elsewhere in the heap. pack_info() read the
same names with %s for SHOW RELAYLOG EVENTS, past the buffer in the
same way.

Bound the lengths in the constructor. It now rejects a Table_map event
whose declared database name, or whose declared table name, exceeds
NAME_LEN, before it sizes the name buffers, so is_valid() returns
false. It also terminates each name it copies, so a source that holds
no null does not leave the name unterminated. A master naming a table
with an oversized name now stops the slave's SQL thread with
ER_SLAVE_RELAY_LOG_READ_FAILURE instead of crashing it.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Sergei Petrunia
Code cleanup in JSON array-of-object reading, add unit tests.
Brandon Nesterenko
MDEV-40644: Slave SQL Thread Overflow on Malformed Table_map_log_event

A slave can crash when its master sends a Table_map event whose table
name is longer than an identifier can be. The SQL thread applying the
event copies the name into a fixed buffer and overruns it, and the SQL
thread crashes.

A Table_map event carries its database and table names each behind a
one-byte length. The constructor that parses the event sizes a buffer
for each name from its declared length, and do_apply_event() copies
both names out of the event into NAME_LEN+1 byte buffers with strmov().
strmov() copies up to the terminating null. A real identifier is at
most NAME_LEN bytes, so the name fits the buffer and carries that null.

The constructor never bounded the two lengths. Each length is read as a
byte, so each can reach 255. The constructor copied each name with
strncpy() over the declared length, which writes no terminating null
when the source holds none. do_apply_event() then copied the name into
its NAME_LEN+1 byte buffer with strmov(), which ran past the buffer
until it reached a null elsewhere in the heap. pack_info() read the
same names with %s for SHOW RELAYLOG EVENTS, past the buffer in the
same way.

Bound the lengths in the constructor. It now rejects a Table_map event
whose declared database name, or whose declared table name, exceeds
NAME_LEN, before it sizes the name buffers, so is_valid() returns
false. It also terminates each name it copies, so a source that holds
no null does not leave the name unterminated. A master naming a table
with an oversized name now stops the slave's SQL thread with
ER_SLAVE_RELAY_LOG_READ_FAILURE instead of crashing it.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Brandon Nesterenko
MDEV-40646: Undersized Table_map Metadata can Crash Slave

A slave can crash or leak heap content when its master sends a Table_map
event that declares columns whose types need more field metadata than
the event carries. Building its description of the table, the slave
reads past the end of the field metadata the event holds. The bytes
beyond the buffer are decoded as column metadata.

A Table_map event carries a column count, one type byte per column,
and a block of field metadata. A column's type fixes how many metadata
bytes it consumes: two for the string, enum, set, bit, varchar, and
decimal types, one for the blob, float, and high precision temporal
types, none for the rest. The master writes exactly that many bytes
per column, so the metadata block a well formed event carries is the
sum over its columns. The slave sizes the metadata block from the
event and, while building its table_def, walks the columns in order
and reads each type's metadata bytes from that block.

The slave checked only that the metadata block was no larger than two
bytes per column. It never checked the block against the column types
themselves. table_def() then walked every declared column and indexed
the metadata block for each, so a column whose type needed metadata
the block did not hold read past the end of it. A column count large
enough carried that read far past the allocation.

The Table_map reader now sums the metadata the declared column types
consume and rejects the event when the block is smaller than that
sum, so is_valid() reports the event as invalid and the SQL thread
stops with a relay log read failure before any column is decoded. A
block larger than the sum is still accepted, because every consumer
reads only the bytes the types call for. A column type this slave
does not recognize counts as zero in the sum, so Table_map events
from a newer master using such a type still parse. A Table_map whose
metadata cannot cover its column types now stops the slave with an
error instead of reading past the buffer.

Note that the regression test does not crash a regular debug build;
an ASAN build is required to show the invalid memory access.  The 24
column table map it injects overruns the metadata allocation by a
bounded amount that lands in mapped heap, which a debug build reads
without faulting and then stops the SQL thread with
ER_SLAVE_CONVERSION_FAILED on the column type mismatch

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Jan Lindström
MDEV-40645 : Slave Crash on Malformed User_var_log_event

A crafted DECIMAL user variable event makes bin2decimal() allocate and
copy a size taken entirely from two attacker-supplied bytes.

bin2decimal() has no parameter telling it how many bytes are available
(strings/decimal.c:1424):

  int bin_size= decimal_bin_size(precision, scale);
  sanity(to);
  d_copy= (uchar*) my_alloca(bin_size);
  memcpy(d_copy, from, bin_size);

decimal_bin_size() guards its inputs with DBUG_ASSERT(precision > 0)
and DBUG_ASSERT(scale <= precision) only, so release builds proceed with
anything. my_alloca is real alloca in this build.

Precision and scale arrive as val[0] and val[1]. Three call sites
pass them through:

* the applier, User_var_log_event::do_apply_event
(sql/log_event_server.cc:4563), which checks only val_len < 3
* pack_info (sql/log_event_server.cc:4356), which checks nothing, not
even val_len >= 3
* mysqlbinlog (sql/log_event_client.cc:2489), which checks nothing
The constructor does bound val_len against the event
(sql/log_event.cc:3064), so val_len is honest, but nothing ties
decimal_bin_size(val[0], val[1]) to it. val_len of 3 with a precision
of 65 already asks for 30 bytes where 1 is available.

Fixed by adding guards against malformed precision, scale and
val_len.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
bsrikanth-mariadb
MDEV-40387: perfschema.misc fails on replay

Disable the testfile, as we don't capture context for performance schema
tables.
bsrikanth-mariadb
MDEV-40389: type_test.type_test_int8 fails on replay

plugins are not yet supported in replay mode.

So, disabling tests type_test.type_test_int8, and type_test.type_test_double
to be run in replay-server mode
Oleksandr Byelkin
Merge branch 'bb-10.11-release' into bb-11.4-release
Brandon Nesterenko
MDEV-40492: Oversized Fake Rotate Event can Crash Slave

A slave can crash when the master's binlog_checksum setting differs
from the checksum the slave's relay log carries. On the first Rotate
event of such a connection, the slave IO thread copies the event onto
a stack buffer sized for the longest binary log file name a master can
send. A master that names a longer file makes the copy run off the end
of that buffer, and what it writes past the end is the file name
itself. A name that overruns the buffer by a few hundred bytes reaches
only the other buffers of the same stack frame, which the slave does
not read on this path, so it keeps running. A name that overruns the
whole frame takes the return address with it, and the slave crashes.
Builds with AddressSanitizer stop at the copy and report a
stack-buffer-overflow.

The fake Rotate event that opens a connection is rewritten when the
master's checksum policy and the relay log's disagree. One case adds a
checksum to the event, and the other strips one. Both copy the event
into rot_buf using the length the master sent, and neither compared
that length against the size of rot_buf. The Rotate_log_event
constructed just above bounds its own copy of the file name at
FN_REFLEN-1, which leaves the length the master sent untouched.
rot_buf was itself one checksum shorter than the event the first case
produces from the longest file name.

Bound the event before either case copies it. queue_event() now
compares the event's length against the longest Rotate a master can
send: the two headers, a file name of at most FN_REFLEN bytes, and the
checksum the master's own policy puts on the event. rot_buf now
reserves that checksum length as well. A Rotate event naming a longer
file stops the slave IO thread with an error reporting the length the
master sent, instead of running off the end of the buffer.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Oleksandr Byelkin
Merge branch '10.11' into bb-10.11-release
Brandon Nesterenko
MDEV-40643: Corrupt Heartbeat Log Event can Crash Slave

A slave can crash when its master sends an event shorter than the
event's own header. A truncated heartbeat makes debug builds fail an
assertion in Binary_string::realloc_raw(). On 64-bit release builds it
stops the slave IO thread with an error message that omits the log
file name. If the master also declares an oversized common header in
its format description event, the slave allocates and fills nearly 4GB
instead. On 32-bit release builds it writes that error message past
the end of a stack buffer. An event under four bytes crashes every
build, because the slave checksums roughly 16 EiB and reads far past
the end of the packet.

Neither queue_event() nor the Heartbeat_log_event constructor bounded
the length the master sent. queue_event() handed that length to
event_checksum_test(), which subtracted the checksum length from it,
and the result wrapped on an event under four bytes. The constructor
computed the log file name length as event_len minus the header
lengths, and on a truncated heartbeat that subtraction wrapped to a
value near 4GB. queue_event() rejected the heartbeat as invalid, so
its error path appended the log file name to the error message using
the wrapped length.

Add the missing bound in both places. queue_event() now rejects an event
shorter than the common header before anything reads that header, and
the slave IO thread stops with an error. The Heartbeat_log_event
constructor now compares event_len against the combined header lengths
before the subtraction. A heartbeat that clears the first check but
still stops inside its headers keeps ident_len at 0 and log_ident at
NULL, so the error path appends nothing. That constructor check also
precedes the read of the extended log position, which previously ran on
a short event before any validation. A short event from the master now
stops the slave IO thread with an error instead of crashing the server.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Brandon Nesterenko
MDEV-40648: Replication Undefined Behavior on Malformed Rotate Log Event

A slave stops with a relay log read failure when its master sends an
event whose header declares a length other than the number of bytes the
event arrived in. The file and position the slave reports for that
failure are not where the problem is. A master that logs no checksum
does not stop the slave at all. Such a master can make the slave run one
statement twice, leaving the slave's data holding a row the master's
binary log never carried. A "malicious" master can already send whatever
events it likes, so what this defeats is comparing a slave's applied
stream against the master's binary log.

The slave IO thread reads each event from the master as one network
packet, and writes that packet into the relay log unchanged, using the
packet's own length. Every later reader of that relay log frames the
events by a different length: the one each event's header declares at
EVENT_LEN_OFFSET. A master writing an event sets the two to the same
value. Log_event::read_log_event(), which parses the events that
queue_event() does not construct itself, checks only that the packet
reaches EVENT_LEN_OFFSET, and never compares the declared length with
the length of the packet.

queue_event() never compared the two lengths either. An event declaring
fewer bytes than the packet held reached the relay log with the extra
bytes behind the event. The SQL thread framed its next read from inside
the previous event. Where the master had placed a complete event in
those extra bytes, and no checksum covered the packet, the SQL thread
applied that second event.

This patch adds validation to ensure the lengths are equal. An event
whose lengths disagree will stop the IO thread with
ER_SLAVE_FATAL_ERROR, and the relay log will never receive the event.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
Brandon Nesterenko
MDEV-40648: Replication Undefined Behavior on Malformed Rotate Log Event

A slave stops with a relay log read failure when its master sends an
event whose header declares a length other than the number of bytes the
event arrived in. The file and position the slave reports for that
failure are not where the problem is. A master that logs no checksum
does not stop the slave at all. Such a master can make the slave run one
statement twice, leaving the slave's data holding a row the master's
binary log never carried. A "malicious" master can already send whatever
events it likes, so what this defeats is comparing a slave's applied
stream against the master's binary log.

The slave IO thread reads each event from the master as one network
packet, and writes that packet into the relay log unchanged, using the
packet's own length. Every later reader of that relay log frames the
events by a different length: the one each event's header declares at
EVENT_LEN_OFFSET. A master writing an event sets the two to the same
value. Log_event::read_log_event(), which parses the events that
queue_event() does not construct itself, checks only that the packet
reaches EVENT_LEN_OFFSET, and never compares the declared length with
the length of the packet.

queue_event() never compared the two lengths either. An event declaring
fewer bytes than the packet held reached the relay log with the extra
bytes behind the event. The SQL thread framed its next read from inside
the previous event. Where the master had placed a complete event in
those extra bytes, and no checksum covered the packet, the SQL thread
applied that second event.

This patch adds validation to ensure the lengths are equal. An event
whose lengths disagree will stop the IO thread with
ER_SLAVE_FATAL_ERROR, and the relay log will never receive the event.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
ARaveala
MDEV-39762: Slave Overflow on Malformed Query_compressed_log_event

A replica can crash when a compressed event from its master carries a
corrupted uncompressed length. A compressed Query event whose length
is near 4GB overwrites the IO thread's stack with the event's content.
A compressed rows event of the same shape makes the replica ask for an
allocation of 4GB. A debug build that gets that memory then fails an
assertion in binlog_buf_uncompress(). On a build where a ulong is 32
bits wide, both events overwrite the IO thread's stack.

The IO thread uncompresses a compressed event before writing the event
to the relay log, and offers the uncompress function 4096 bytes of its
own stack to hold the result. The function computes how large the event
will be once uncompressed, and that size decides where the uncompressed
event goes. An event that fits in the stack buffer is uncompressed
there. A larger event is uncompressed into an allocation the function
makes at that size. With either buffer, the function tells zlib that
the room available is the uncompressed length the event declared.

query_event_uncompress() never bounded the length read out of the
event. The function added the header length to that value, rounded the
sum up for alignment, and cast the result to uint32. The cast dropped
the high bits of a sum above 4GB, so a length near 4GB produced a size
of a few dozen bytes. That size fit the stack buffer, so content of any
size went onto the IO thread's stack. row_log_event_uncompress()
computes that sum in a ulong, which wraps where a ulong is 32 bits
wide. The Query_compressed_log_event constructor and
Rows_log_event::uncompress_buf() size their allocations from the same
unbounded length.

Reject an uncompressed length above MAX_MAX_ALLOWED_PACKET at every
point the length is read, before any size is computed from it. No
master writes a larger length, because max_allowed_packet is capped at
1GB. query_event_uncompress() also keeps that size in a size_t, so the
allocation is made from the whole computed value. Both functions return
the error code for the IO thread to report, so a length over the limit
stops the IO thread with ER_TOO_BIG_FOR_UNCOMPRESS rather than the
generic uncompress error. A compressed event with a corrupted
uncompressed length now stops the replica's IO thread with an error
instead of crashing the replica.

Alexandra Raveala wrote the original patch and its first regression
test, contributed through PR 5413. Brandon Nesterenko saw the work
through for the 10.6 release, replacing the test with one that makes
the master write the corrupt length and reworking how the IO thread
reports the error.

Co-authored-by: Brandon Nesterenko <[email protected]>
Reviewed-by: Kristian Nielsen <[email protected]>
Oleksandr Byelkin
Merge fix.
bsrikanth-mariadb
MDEV-40388: sequence.simple fails on replay

The problem is that, when recording is enabled for the query such as,
explain select * from seq_1_to_10;
it recorded the table context having a DDL definition as: -

CREATE TABLE `seq_1_to_10` (
    ->  `seq` bigint(20) unsigned NOT NULL,
    ->  PRIMARY KEY (`seq`)
    -> ) ENGINE=SEQUENCE DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci;

Now, when that context is replayed, the DDL statement is executed.
But, we cannot create such a table, and instead it errors out saying
ERROR 1050 (42S01): Table 'seq_1_to_10' already exists.

Solution is to use: -
  CREATE TABLE IF NOT EXISTS seq_1_to_10 ...;

=====

Also, there is a different way to use sequences as: -
  Create sequence s1;
  Explain select * from s1;

Here, we should be recording the DDL statement, but no need to store the
stats for it. However, we didn't record the DDL statement earlier.
Moreover, sequence's next value should be the same in the replay environment.

Solution here is to record the DDL for such a sequence as
  CREATE TABLE IF NOT EXISTS s1 ...;
and also set its start value as the recorded environment's previous value using
  SELECT SETVAL(s1, prev_value);
Sergei Petrunia
Add comments about KEY_PART_INFO::type, KEY_PART_INFO::key_type,

... and Column_definition_attributes::pack_flag.

Approved-by: Dave Gosselin ([email protected])
bsrikanth-mariadb
MDEV-40518: add both drop table and view stmts

The context only stored "DROP TABLE IF EXISTS t1" before adding a
"CREATE TABLE t1" statement. However, there can be a view named t1
already existing in the database. When the context was replayed,
the CREATE statement failed stating t1 already exists.

Solution is to add both "DROP TABLE IF EXISTS t1", and
"DROP VIEW IF EXISTS t1" before adding a
"CREATE TABLE t1" statement into the context.
Brandon Nesterenko
MDEV-40646: Undersized Table_map Metadata can Crash Slave

A slave can crash or leak heap content when its master sends a Table_map
event that declares columns whose types need more field metadata than
the event carries. Building its description of the table, the slave
reads past the end of the field metadata the event holds. The bytes
beyond the buffer are decoded as column metadata.

A Table_map event carries a column count, one type byte per column,
and a block of field metadata. A column's type fixes how many metadata
bytes it consumes: two for the string, enum, set, bit, varchar, and
decimal types, one for the blob, float, and high precision temporal
types, none for the rest. The master writes exactly that many bytes
per column, so the metadata block a well formed event carries is the
sum over its columns. The slave sizes the metadata block from the
event and, while building its table_def, walks the columns in order
and reads each type's metadata bytes from that block.

The slave checked only that the metadata block was no larger than two
bytes per column. It never checked the block against the column types
themselves. table_def() then walked every declared column and indexed
the metadata block for each, so a column whose type needed metadata
the block did not hold read past the end of it. A column count large
enough carried that read far past the allocation.

The Table_map reader now sums the metadata the declared column types
consume and rejects the event when the block is smaller than that
sum, so is_valid() reports the event as invalid and the SQL thread
stops with a relay log read failure before any column is decoded. A
block larger than the sum is still accepted, because every consumer
reads only the bytes the types call for. A column type this slave
does not recognize counts as zero in the sum, so Table_map events
from a newer master using such a type still parse. A Table_map whose
metadata cannot cover its column types now stops the slave with an
error instead of reading past the buffer.

Note that the regression test does not crash a regular debug build;
an ASAN build is required to show the invalid memory access.  The 24
column table map it injects overruns the metadata allocation by a
bounded amount that lands in mapped heap, which a debug build reads
without faulting and then stops the SQL thread with
ER_SLAVE_CONVERSION_FAILED on the column type mismatch.

Reviewed-by: Kristian Nielsen <[email protected]>
Signed-off-by: Brandon Nesterenko <[email protected]>
bsrikanth-mariadb
MDEV-40383:innodb_gis.point_basic fails on replay

There are 2 problems: -
1. The REPLACE statement that is recorded doesn't store the
  value of geometry type field correctly.
2. The table definition that got recorded has fields with non-null constraint,
  and no default value is specified.
  Also, the "REPLACE INTO" statement that gets stored in the context,
  doesn't have any value specified for these non-null fields.

Solution is to: -
1. When using REPLACE INTO statement, store all the non-numeric values in HEX,
  whenever conversion from field's charset to output's charset is lossy.
2. Instead of storing only the column values that were projected in the
  query, store all the non-virtual column values into the recorded
  REPLACE INTO statement.

Implementation details: -
1. Introduce a new method is_charset_conversion_lossless() in filesort.cc,
  to check if the output charset to which field's data is being written to,
  results in a lossless conversion. If so, non-numeric values being witten
  using REPLACE INTO statement are stored in string representation,
  else they are converted to HEX.
2. Modify join_read_const(), and join_read_system() methods in sql_select.cc,
  and opt_sum_query() method in opt_sum.cc the following way: -
    a. Extend the read_set to make sure, we read all the non-virtual column
        using Optimizer_context_recorder::prepare_captured_row_read().
        This method also saves the original read_set.
    b. Read the row.
    c. Dump the row into the context when no error is noticed while
        reading. Irrespective of the error, restore back the read_set state to
        the original using Optimizer_context_recorder::finish_captured_row_read()
Alexander Barkov
MDEV-40722 DROP PACKAGE leaves PACKAGE BODY grant in mysql.procs_priv

DROP PACKAGE now removes both PACKAGE BODY and PACKAGE privileges
for the given package.
bsrikanth-mariadb
MDEV-40220: Add server version to optimizer context

Additionaly, include Version_source_revision as well.
These are read only informative sya variables. So, they are included as
comments instead of SET commands.
Alexander Barkov
MDEV-39587 Package-wide TYPE for variable declarations

SET sql_mode=ORACLE;
DELIMITER $$
CREATE OR REPLACE PACKAGE pkg AS
  -- Declare a package public data type
  TYPE varchar_array IS TABLE OF VARCHAR(2000) INDEX BY INTEGER;
END;
$$
DELIMITER ;
DELIMITER $$

CREATE OR REPLACE PROCEDURE p1 AS
  v pkg.varchar_array; -- Use the package public data type
BEGIN
  v(0):='test';
  SELECT v(0);
END;
$$
DELIMITER ;

Note, the change is done only for sql_mode=ORACLE, because the TYPE
declaration is not available for the default mode.

Where package-wide types are available
--------------------------------------
- Variabe list type:
    DECLARE var pkg1.type1;

- RETURN type for a package routine:
    CREATE FUNCTION .. RETURN pkg1.type1 ...

- Parameter type for a package routine:
    PROCEDURE p1(param1 pkg1.type1);

- Assoc array element type:
    TYPE assoc1_t IS TABLE OF pkg1.type1 ...

- REF CURSOR RETURN type:
    TYPE cur1_t IS REF CURSOR RETURN pkg1.type1;

Change details
--------------

- Adding a member Lex_length_and_dec_st::m_foreign_module_type
  It's set to true when the data type was initialized from a TYPE
  in foreign routine (e.g. in PACKAGE spec).
  It's needed to prevent use of qualified identifiers in public contexts,
  i.e. in schema public routine parameter types and schema publuc function
  RETURN types.
  Adding a helper method sp_head::check_applicability() which prevents
  use of qualified types in public context.

- Adding a helper method sp_head::raise_unknown_data_type().

- Adding methods LEX::set_field_type_typedef_package_spec() for
  2-step and 3-step qualified indentifiers.
  It's used in field_type_all_with_typedefs which covers cases:
  - Variabe list type        : DECLARE var pkg1.type1;
  - RETURN type              : CREATE FUNCTION .. RETURN pkg1.type1 ...
  - Parameter type          : PROCEDURE p1(param1 pkg1.type1);
  - Assoc array element type : TYPE assoc1_t IS TABLE OF pkg1.type1 ...

- Adding a method LEX::declare_type_ref_cursor_return_typedef().
  It handles cases when a new TYPE REF CURSOR RETURN is declared,
  for both for qualified RETURN types and non-qualified RETURN types:
  - TYPE cur0_t IS REF CURSOR RETURN rec1_t;
  - TYPE cur0_t IS REF CURSOR RETURN pkg1.rec1_t;
  - TYPE cur0_t IS REF CURSOR RETURN db1.pkg1.rec1_t;

  The code was moved from LEX::declare_type_ref_cursor() into
  LEX::declare_type_ref_cursor_return_typedef() and extended
  to cover qualified RETURN types.

- Adding a method Sql_path::find_package_spec_type().
  It iterates through all schemas specified in @@path and searches
  for the given type in the given package.

- Adding a helper method sp_pcontext::type_defs_add_ref_cursor()
  to reuse the code.

- Adding a new method sp_package::get_typedef() to search
  for TYPE definitions in PACKAGE specifications.

- Adding a new method sp_head::get_typedef_package_spec()
  to search for TYPE definitions used by a PROCEDURE or FUNCTION.

- Adding a helper method
    Sp_handler::sp_cache_routine_reentrant_suppress_errors
  Adding a method Sp_handler::find_package_spec().
Oleksandr Byelkin
Merge branch 'bb-10.11-release' into bb-11.4-release
Daniel Black
MDEV-40165 JSON_EQUAL/JSON_NORMALIZE/JSON_CONTAINS error handling

Adds test cases that validate the incorrect null handling
in the 2894e90b6545d3c7fa13ff00fb50c9e8631003e7 commit
and other JSON_EQUALS,CONTAINS/OVERLAPS behaviour.