Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Categories: connectors experimental galera main
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

connectors experimental galera main
Monty
MDEV-38246 aria_read index failed on encrypted database during backup

The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.
Nikita Malyavin
MDEV-38448 assertion ...tdc->ref_count > 0 failed on LOCK gtt+open error

If open_global_temporary_table failed, a goto was to an incorrect place.
In locked_tables_mode, an extra acquire didn't happen, so a release on
error shouldn't have happened as well (i.e. goto err_lock).
That is, in locked_tables_mode, just return immediately on error.

The easiest way to reproduce it was with DML while pseudo_slave_mode=1,
but actually any error inside open_global_temporary_table would do.
Sergei Golubchik
fix tests for --ps: GTT
Raghunandan Bhat
MDEV-38487: Prevent aggregate function cloning when pushing HAVING into WHERE

Problem:
  When building a pushable condition that can be pushed from HAVING into
  WHERE, the server tries to clone aggregate functions. This unnecessary
  because aggregate functions will not be pushed into WHERE anyway.

Fix:
  This fix introduces a check within `Item::build_pushable_cond` to skip
  cloning aggregate functions.

Also fixes assert failure in MDEV-38492, by adding a missing copy method
for `Item_aggregate_ref`.
Vladislav Vaintroub
MDEV-38498 mariadb-dump -dir doesn't convert database names

Similar to MDEV-37483, use file name encoding for dbnames to create
directories. Adapt mariadb-import to convert the names back.
Lawrin Novitsky
Merge branch 'cpp-1.1'
Nikita Malyavin
MDEV-38480 GTT: Aria gets error HA_ERR_CRASHED_ON_USAGE after TRUNCATE

Do not call HA_EXTRA_PREPARE_FOR_DROP for the parent table
Monty
squash! 7b54e0496526d366d4acc16530ea215b06ae92f6

MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
          table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge

Other things:
- EITS data is preserved if create or replace fails if
  drop_before_create_or_replace=OFF. If ON, then create or replace
  will drop EITS before the drop of the original table (as before).
Lawrin Novitsky
Merge branch 'cpp-1.0' into cpp-1.1
Michael Widenius
MDEV-19683 Add support for Oracle TO_DATE()

Syntax:
TO_DATE(string_expression [DEFAULT string_expression ON CONVERSION ERROR],
        format_string [,NLS_FORMAT_STRING])
The format_string has the same format elements as TO_CHAR(), except a
few elements that are not supported/usable for TO_DATE().
TO_DATE() returns a datetime or date value, depending on if the format
element FF is used.

Allowed separators, same as TO_CHAR():
space, tab and any of !#%'()*+,-./:;<=>

'&' can also be used if next character is not a character a-z or A-Z
"text' indicates a text string that is verbatim in the format. One cannot
use " as a separator.

Format elements supported by TO_DATE():
AD          Anno Domini ("in the year of the Lord")
AD_DOT      Anno Domini ("in the year of the Lord")
AM          Meridian indicator (Before midday)
AM_DOT      Meridian indicator (Before midday)
DAY        Name of day
DD          Day (1-31)
DDD        Day of year (1-336)
DY          Abbreviated name of day
FF[1-6]    Fractional seconds
HH          Hour (1-12)
HH12        Hour (1-12)
HH24        Hour (0-23)
MI          Minutes (0-59)
MM          Month (1-12)
MON        Abbreviated name of month
MONTH      Name of Month
PM          Meridian indicator (After midday)
PM_DOT      Meridian indicator (After midday)
RR          20th century dates in the 21st century. 2 digits
            50-99 is assumed from 2000, 0-49 is assumed from 1900.
RRRR        20th century dates in the 21st century. 4 digits
SS          Seconds
SYYYY      Signed 4 digit year; MariaDB only supports positive years
Y          1 digit year
YY          2 digits year
YYY        3 digits year
YYYY        4 digits year

Note that if there is a missing part of the date, the current date is used!
For example if 'MM-DD HH-MM-SS' then the current year will be used.
(Oracle behaviour)

Not supported options:
BC, D, DL, DS, E, EE, FM, FX, RM, SSSSS, TS, TZD, TZH, TZR, X,SY
BC is not supported by MariaDB datetime.
Most of the other are exotic formats, format modifies other things that
does not make in MariaDB as we return datetime or datetime with fractions,
not string.
D (day-of-week) is not supported as it is not clear exactly how it would
map to MariaDB. This element depends on the NLS territory of the session.

Extensions / differences compared to Oracle;
- MariaDB supports FF (fractional seconds).  If FF[£] is used,
  then TO_DATE will return a datetime with # of subseconds.
  If FF is not used a datetime will be returned.
- Names can be shortened to it's unique prefix. For example January and Ja
  works fine.
- No error if the date string is shorter format_string. This is useful to
  get a date from a mixed set of strings in date or datetime format.
  Oracle gives an error if date string is too short.
- MariaDB supports short locales as language names
- NLS_DATE_FORMAT can use both " and ' for quoting.

New formats handled by TO_CHAR():
FF[1-6]    Fractional seconds
DDD        Daynumber 1-366
IW          Week 1-53 according to ISO 8601
I          1 digit year according to ISO 8601
IY          2 digit year according to ISO 8601
IYY        3 digit year according to ISO 8601
IYYY        4 digit year according to ISO 8601
SYYY        4 digit year according to ISO 8601 (Oracle can do signed)

Supported NLS_FORMAT_STRING options are:
NLS_CALENDAR=GREGORIAN
NLS_DATE_LANGUAGE=language

Support languages are:
- All MariaDB short locales, like en_AU.
- The following Oracle language names:
ALBANIAN, AMERICAN, ARABIC, BASQUE, BELARUSIAN, BRAZILIAN PORTUGUESE
BULGARIAN, CANADIAN FRENCH, CATALAN, CROATIAN, CYRILLIC SERBIAN CZECH,
DANISH, DUTCH, EGYPTIAN, ENGLISH, ESTONIAN, FINNISH, FRENCH, GERMAN,
GREEK, HEBREW, HINDI, HUNGARIAN, ICELANDIC, INDONESIAN ITALIAN,
JAPANESE, KANNADA, KOREAN, LATIN AMERICAN SPANISH, LATVIAN,
LITHUANIAN, MACEDONIAN, MALAY, MEXICAN SPANISH, NORWEGIAN, POLISH,
PORTUGUESE, ROMANIAN, RUSSIAN, SIMPLIFIED CHINESE, SLOVAK, SLOVENIAN,
SPANISH, SWAHILI, SWEDISH, TAMIL, THAI, TRADITIONAL CHINESE, TURKISH,
UKRAINIAN, URDU, VIETNAMESE

Development bugs fixed:
MDEV-38403 Server crashes in Item_func_to_date::fix_length_and_dec upon
          using an invalid argument
MDEV-38400 compat/oracle.func_to_date fails with PS protocol and cursor
          protocol (Fixed by Serg)
MDEV-38404 TO_DATE: MTR coverage omissions, round 1
MDEV-38509 TO_DATE: AD_DOT does not appear to be supported
MDEV-38513 TO_DATE: NULL value for format string causes assertion failure
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

Fix some tests. Make ext_buf_pool test more stable avoiding race
conditions for read/write counters.

The code is dirty, needs clean up.
Kristian Nielsen
MDEV-38465: Savepoint in trigger causes transactional inconsistency

SAVEPOINT inside a trigger doesn't work correctly. Setting a savepoint
inside a trigger somehow loses the implicit savepoint set at transaction
start, so that the partial changes are left if the statement later fails.
Referencing an existing savepoint claims the savepoint does not exist (and
it is in any case very unclear what exactly it should mean to rollback to a
savepoint from the middle of a statement, or set in the middle of a prior
statement).

These problems are independent of binlog-in-engine, but in the new binlog
implementation we are trying to make things work more correctly and
robustly, so let's disallow use of savepoints inside triggers. The new
binlog is off by default, so backwards compatibility is less of a concern,
though arguably disallowing savepoints in triggers would be better done
unconditionally.

Signed-off-by: Kristian Nielsen <[email protected]>
Raghunandan Bhat
MDEV-38487: Prevent aggregate function cloning when pushing HAVING into WHERE

Problem:
  When building a pushable condition that can be pushed from HAVING into
  WHERE, the server tries to clone aggregate functions. This unnecessary
  because aggregate functions will not be pushed into WHERE anyway.

Fix:
  This fix introduces a check within `Item::build_pushable_cond` to skip
  cloning aggregate functions.

Also fixes assert failure in MDEV-38492, by adding a missing copy method
for `Item_aggregate_ref`.
Nikita Malyavin
MDEV-37898 No GTT MTR coverage on release builds due to debug_sync
requirement

Deny temporary tables creation for sphinx.
Oleg Smirnov
MDEV-35880: Fix incorrect error message when MAX_EXECUTION_TIME was exceeded

The error message shown when MAX_EXECUTION_TIME hint causes a timeout
incorrectly referred to "max_statement_time", which is a different
variable with different units (seconds vs milliseconds). This could
confuse users.

Before: "Query execution was interrupted (max_statement_time exceeded)"
After:  "Query was interrupted: execution time limit 1.5 sec exceeded"

The new message is neutral and accurate for both max_statement_time
system variable and MAX_EXECUTION_TIME optimizer hint, displaying the
actual timeout value that was exceeded.
Lawrin Novitsky
CONCPP-149 Introducing Date, Time and Timestamp classes

and corresponding methods.
The classes are made for representation of the corresponding database
types and for easier for application manipulation of that data - before
the only possibility was work with string representation.
Classes have construtors from string, set of integers representing each
field, std::chrono::time_point<system_clock>, setters and getters, and
comparison methods - after, before equals and compareTo
ResultSet getters return objects of corresponding class - they are not
abstract but defined almost completely in the connector.
Monty
Update Galera test results
Daniel Bartholomew
bump the VERSION
Nikita Malyavin
MDEV-38468 GTT: use-after-free after failed BINLOG event in XA mode

Deny setting pseudo_slave_mode for GTT even when rli_fake is set.
Oleg Smirnov
MDEV-36686: Fix crash with DEFAULT() in WHERE clause with nested views

Item_default_value::do_build_clone() was performing a shallow copy,
causing the cloned item to share its 'arg' pointer with the original.

During condition pushdown for derived tables:
1. build_pushable_cond() cloned the WHERE clause condition
2. grouping_field_transformer_for_where transformed the cloned
  DEFAULT(a) to reference the inner select's grouping field,
  modifying the shared 'arg' via change_item_tree()
3. cleanup_excluding_const_fields_processor cleaned up the transformed
  items, setting Item_field::field = 0
4. The original WHERE clause still referenced the cleaned-up item
5. Crash in Item_field::used_tables() accessing field->table

Fix: Make do_build_clone() create a deep clone of 'arg' member,
ensuring the cloned Item_default_value has independent state.
Monty
MDEV-38246 aria_read index failed on encrypted database during backup

The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.

Other things:
- Fixed a (not critical) memory leak in mariabackup
Monty
MDEV-38246 aria_read index failed on encrypted database during backup

The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.

Other things:
- Fixed some (not critical) memory leaks in mariabackup
Kristian Nielsen
Binlog-in-engine: MDEV-38462: Large binlog write can overwrite head of InnoDB redo log

The code for binlogging out-of-band data was missing an appropriate call to
log_free_check(). This call is needed to throttle write activity and wait
for an InnoDB checkpoint, when the redo log is too small (or otherwise has
insufficient space available) to accomodate the write activity.

Signed-off-by: Kristian Nielsen <[email protected]>
Rex Johnston
MDEV-38347 Debugging functions

Extend current dbug_print* functions in such a way as to facilitate easy
high level debugging.  Introduce a single overloaded print function
defined in dbp.h, DBUG_PRINT_FUNCTION, settable to your preference.
Vladislav Vaintroub
MDEV-37483 mariadb-dump -dir doesn't convert database names

Similar to MDEV-37483, use file name encoding for dbnames to create
directories. Adapt mariadb-import to convert the names back.
Raghunandan Bhat
MDEV-38487: Prevent aggregate function cloning when pushing HAVING into WHERE

Problem:
  When building a pushable condition that can be pushed from HAVING into
  WHERE, the server tries to clone aggregate functions. This unnecessary
  because aggregate functions will not be pushed into WHERE anyway.

Fix:
  This fix introduces a check within `Item::build_pushable_cond` to skip
  cloning aggregate functions.

Also fixes assert failure in MDEV-38492, by adding a missing copy method
for `Item_aggregate_ref`.
Raghunandan Bhat
MDEV-38487: Avoid cloning aggregate functions for HAVING to WHERE pushdown

Problem:
  When building a pushable condition that can be pushed from HAVING into
  WHERE, the server tries to clone aggregate functions. This unnecessary
  because aggregate functions will not be pushed into WHERE anyway.

Fix:
  This fix introduces a check within `Item::build_pushable_cond` to skip
  cloning aggregate functions.
Nikita Malyavin
MDEV-37898 No release GTT MTR tests due to debug_sync injection
Vladislav Vaintroub
MDEV-38498 mariadb-dump -dir doesn't convert database names

Similar to MDEV-37483, use file name encoding for dbnames to create
directories. Adapt mariadb-import to convert the names back.
Rex Johnston
MDEV-38473 Incorrect Empty Set with HAVING clause when SELECT and GROUP BY use different aliases for the same column

MDEV-29300 fix causes a wrong result by incorrect removing a wrapper to
an item that needed to be wrapped for the correct result.  Direct access
to the item causes an incorrect table reference to be used during
join evaluation.  We reverting that fix.

Our original problem query is this
SELECT (SELECT 0 GROUP BY c1 HAVING (SELECT c1)) FROM t1 group by c1;

JOIN::prepare on
/* select#2 */ select 0 group by t1.t1a having (subquery#3)

fixing t1.t1a in group by clause, calls fix_outer_field()
this item is resolved in an outer select (#1) and it is a grouping select,
so we wrap it in Item_outer_ref and set this item to unfixed for later
fixing in fix_inner_refs().

JOIN::prepare continues onto the having clause and fixes (subquery#3) which
calls initiates the prepare series of calls, leading to setup_fields on the
fields in this JOIN, one of which is an outer reference t1a.
This is resolved to the item in the next most outer select in the group by
clause. This item has been wrapped with an unfixed Item_outer_ref.
It is found in resolve_ref_in_select_and_group() is it expected that
this item will have already been fixed, hence this call in

Item_field::fix_outer_field()

DBUG_ASSERT(*ref && (*ref)->fixed());

but as explained above, it isn't fixed and debug builds assert here.

Because this wrapper cannot be resolved here for reasons detailed in
fix_inner_refs, and we cannot remove this wrapper without potentially
returning an incorrect result, we have to relax this assertion.
Giorgio Caculli
required <cstdint> inclusion in various headers
Monty
fixup! bfeb39d07208ea5cd56ef818421772ea8bb412a3

MDEV-38521 TO_DATE: Date strings with non-ASCII symbols cause warnings
          and wrong results
Vlad Lesin
MDEV-31956 SSD based InnoDB buffer pool extension

In one of the practical cloud MariaDB setups, a server node accesses its
datadir over the network, but also has a fast local SSD storage for
temporary data. The content of such temporary storage is lost when the
server container is destroyed.

The commit uses this ephemeral fast local storage (SSD) as an extension of
the portion of InnoDB buffer pool (DRAM) that caches persistent data
pages. This cache is separated from the persistent storage of data files
and ib_logfile0 and ignored during backup.

The following system variables were introduced:

innodb_extended_buffer_pool_size - the size of external buffer pool
file, if it equals to 0, external buffer pool will not be used;

innodb_extended_buffer_pool_path - the path to external buffer pool
file.

If innodb_extended_buffer_pool_size is not equal to 0, external buffer
pool file will be created on startup.

Only clean pages will be flushed to external buffer pool file. There is
no need to flush dirty pages, as such pages will become clean after
flushing, and then will be evicted when they reach the tail of LRU list.

The general idea of this commit is to flush clean pages to external
buffer pool file when they are evicted.

A page can be evicted either by transaction thread or by background
thread of page cleaner. In some cases transaction thread is waiting for
page cleaner thread to finish its job. We can't do flushing in external
buffer pool file when transaction threads are waithing for eviction,
that would heart performance. That's why the only case for flushing is
when page cleaner thread evicts pages in background and there are no
waiters. For this purprose buf_pool_t::done_flush_list_waiters_count
variable was introduced, we flush evicted clean pages only if the
variable is zeroed.

Clean pages are evicted in buf_flush_LRU_list_batch() to keep some
amount of pages in buffer pool's free list. That's why we flush every
second page to external buffer pool file, otherwise there could be not
enought amount of pages in free list to let transaction threads to
allocate buffer pool pages without page cleaner waiting. This might be
not a good solution, but this is enought for prototyping.

External buffer pool page is introduced to store information in buffer
pool page hash about the certain page can be read from external buffer
pool file. The first several members of such page must be the same as the
members of internal page. External page frame must be equal to the
certain value to disthinguish external page from internal one. External
buffer pages are preallocated on startup in external pages array. We
could get rid of the frame in external page, and check if the page's
address belongs to the array to distinguish external and internal pages.

There are also external pages free and LRU lists. When some internal page
is decided to be flushed in external buffer pool file, a new external
page is allocated eighter from the head of external free list, or from
the tail of external LRU list. Both lists are protected with
buf_pool.mutex. It makes sense, because a page is removed from internal
LRU list during eviction under buf_pool.mutex.

Then internal page is locked and the allocated external page is attached
to io request for external buffer pool file, and when write request is
completed, the internal page is replaced with external one in page hash,
external page is pushed to the head of external LRU list and internal
page is unlocked. After internal page was removed from external free list,
it was not placed in external LRU, and placed there only
after write completion, so the page can't be used by the other threads
until write is completed.

Page hash chain get element function has additional template parameter,
which notifies the function if external pages must be ignored or not. We
don't ignore external pages in page hash in two cases, when some page is
initialized for read and when one is reinitialized for new page creating.

When an internal page is initialized for read and external page with the
same page id is found in page hash, the internal page is locked,
the external page in replaced with newly initialized internal page in the
page hash chain, the external page is removed from external LRU list and
attached to io request to external buffer pool file. When the io request
is completed, external page is returned to external free list,
internal page is unlocked. So during read external page absents in both
external LRU and free lists and can't be reused.

When an internal page is initialized for new page creating and external
pages with the same page id is found in page hash, we just remove external
page from the page hash chain and external LRU list and push it to the
head of external free list. So the external page can be used for future
flushing.

The pages are flushed to and read from external buffer pool file with
the same manner as they are flushed to their spaces, i.e. compressed and
encrypted pages stay compressed and encrypted in external buffer pool
file.
Jan Smutný
fsp0fsp.cc: log unexpected table in sys tablespace

Fix log message about unexpected table in system tablespace as the current
message can be missleading due to still existing (but already deprecated)
system tables SYS_DATAFILES and SYS_TABLESPACES, reported in
- MDEV-38412

Also adding the informative message with table name of the unexpected table
in system table space.
Monty
squash! 7b54e0496526d366d4acc16530ea215b06ae92f6

MDEV-36501 EITS data is lost after failed attempt to CREATE OR REPLACE
          table
MDEV-36493 Atomic CREATE OR REPLACE ... SELECT blocks InnoDB purge

Other things:
- EITS data is preserved if create or replace fails if
  drop_before_create_or_replace=OFF. If ON, then create or replace
  will drop EITS before the drop of the original table (as before).
Dylan Liu
fix: correct CMake path configuration error that caused build failures when used as a subdirectory.
Oleg Smirnov
MDEV-35880: Fix incorrect error message when MAX_EXECUTION_TIME was exceeded

The error message shown when MAX_EXECUTION_TIME hint causes a timeout
incorrectly referred to "max_statement_time", which is a different
variable with different units (seconds vs milliseconds). This could
confuse users.

Before: "Query execution was interrupted (max_statement_time exceeded)"
After:  "Query was interrupted: execution time limit 1.5 sec exceeded"

The new message is neutral and accurate for both max_statement_time
system variable and MAX_EXECUTION_TIME optimizer hint, displaying the
actual timeout value that was exceeded.
Vladislav Vaintroub
MDEV-38498 mariadb-dump -dir doesn't convert database names

Similar to MDEV-37483, use file name encoding for dbnames to create
directories. Adapt mariadb-import to convert the names back.
Oleg Smirnov
MDEV-35880: Fix incorrect error message when MAX_EXECUTION_TIME was exceeded

The error message shown when MAX_EXECUTION_TIME hint causes a timeout
incorrectly referred to "max_statement_time", which is a different
variable with different units (seconds vs milliseconds). This could
confuse users.

Before: "Query execution was interrupted (max_statement_time exceeded)"
After:  "Query was interrupted: execution time limit 1.5 sec exceeded"

The new message is neutral and accurate for both max_statement_time
system variable and MAX_EXECUTION_TIME optimizer hint, displaying the
actual timeout value that was exceeded.
Aleksey Midenkov
MDEV-32724 Segmentation fault due to Deep Recursion in table.cc and sql_lex.cc

Recursive CTE wrongly detected anchor because inner non-recursive CTE
wrongly assigned with-element table to itself due to inner-outer name
clash.