Страницы

Показаны сообщения с ярлыком sql_macro. Показать все сообщения
Показаны сообщения с ярлыком sql_macro. Показать все сообщения

суббота, 1 февраля 2020 г.

Tracing SQL Macro Expansion

I decided to take a look at what events Oracle provides to diagnose or troubleshoot SQL Macro Expansions.
For that, I took almost the same code that I used before: SQL Macro script
drop table t;
drop table t1;
drop table t2;

CREATE OR REPLACE FUNCTION sample(t DBMS_TF.Table_t, how_many number DEFAULT 5)
RETURN VARCHAR2 SQL_MACRO
AS 
BEGIN 
  RETURN q'[SELECT * 
              FROM t
      WHERE rownum <= how_many]';
END sample;
/

create table t
as
select n
  from xmltable('1 to 10'
  columns
    n int path '.');

create table t1
as
select n,
       'val_'||to_char(n, 'fm00') val
  from xmltable('1 to 10'
  columns
    n int path '.');

create table t2
as
select 'val_'||to_char(n, 'fm00') s
  from xmltable('1 to 10'
  columns
    n int path '.');

select * from t1;
select * from t2;

select *
  from sample(t1, 3);

select *
  from sample(t1);

select *
  from sample(t2);
A query failed when I was trying to supply the table T - which is the same that is used in the SQL macro's definition:
SQL> select *
  2    from sample(t);
  from sample(t)
               *
ERROR at line 2:
ORA-32039: recursive WITH clause must have column alias list
The errorstack dump for ORA-32039 (level=3) did not provide much information about the underlying cause:
----- Error Stack Dump -----
<error barrier> at 0x7ffd69f7aaf0 placed dbkda.c@296
ORA-32039: recursive WITH clause must have column alias list
----- Current SQL Statement for this session (sql_id=d3ktrfsd4s0sr) -----
select * from sample(t)
SQL Trace was referring to the same statement:
PARSE ERROR #140320112869816:len=24 dep=0 uid=119 oct=3 lid=119 tim=1054524149653 err=32039
select * from sample(t)
It showed the following statement, though:
PARSING IN CURSOR #140320116348624 len=446 dep=1 uid=119 oct=47 lid=119 tim=1054472624364 hv=3547957408 ad='7cda6688' sqlid='9awjhv79rm250'

declare
t0 DBMS_TF.Table_t := DBMS_TF.Table_t(column => :0);

begin
:macro_text := "SAMPLE"(t0);
end;
While studying those qksptfSQM functions, namely: qksptfSQM_GetTxt and qksptfSQM_Template, I found that they are calling to the PTF_Comp UTS component (Polymorphic Table Functions Compilation (qksptf)):
   0x000000000f75c87a <+4202>:  mov    $0x6000,%ecx
   0x000000000f75c87f <+4207>:  mov    $0x205018b,%esi
   0x000000000f75c884 <+4212>:  mov    $0x2,%edx
   0x000000000f75c889 <+4217>:  mov    -0x200(%rbp),%r8
   0x000000000f75c890 <+4224>:  mov    -0xcb58(%rbx),%rdi
   0x000000000f75c897 <+4231>:  callq  0x485cef0 <dbgtCtrl_intEvalCtrlEvent>
Therefore, I enabled PTF_Comp tracing:
alter session set events 'trace[ptf_comp]' tracefile_identifier=ptf_comp;
Here is what I got in the trace file:
qksptfSQM_GetTxt(): Anonymous Block
===================================

declare
t0 DBMS_TF.Table_t := DBMS_TF.Table_t(column => :0);

begin
:macro_text := "SAMPLE"(t0);
end;
qksptfSQM_GetTxt(): Macro Text
==============================

SELECT *
              FROM t
             WHERE rownum <= how_many
qksptfSQM_Template(): Template Text
===================================

with  T as (select  /*+ INLINE */ * from "MY_USER"."T")
select "SYS__$".*
from (select 5 "HOW_MANY" from SYS.DUAL) "SAMPLE",
     lateral(SELECT *
              FROM t
             WHERE rownum <= how_many) "SYS__$"
Obviously, the ORA-32039 error is absolutely legit in this case and the PTF_Comp tracing clearly shows where the issue is. It highlights one of the current limitations of that functionality in Oracle 19.6. Beside it demonstrates how literals (HOW_MANY in the example above) are passed through a subquery and joined to a lateral view - that approach has certain restrictions. Apparently there seem to be more restrictions than that according to $ORACLE_HOME/plsql/mesg/pcmus.msg:
776, 0, "SQL macro can only return character return types"
// INDEX: "SQL macro"
// RELEASE: 20.1
// CAUSE: A SQL macro function returned a type that was not a character type like VARCHAR2.
// ACTION:  Change the return type to one of the character types.
777, 0, "scalar SQL macro cannot have argument of type DBMS_TF.TABLE_T"
// INDEX: "SQL macro"
// RELEASE: 20.1
// CAUSE: DBMS_TF.TABLE_T argument was used for scalar macros.
// ACTION:  Change the argument type to a type other than
//          DBMS_TF.TABLE_T.
778, 0, "PARTITION BY, ORDER BY, CLUSTER BY, or PARALLEL_ENABLE are not allowed for SQL macros"
// MANUAL: partition by, order by, cluster by, deterministic, authid or parallel_enable
//         not allowed for SQL macros
// INDEX: "SQL macro"
// RELEASE: 20.1
// CAUSE: An attempt was made to use the PARTITION BY, ORDER BY, CLUSTER BY,
//        DETERMINISTIC, AUTHID, or PARALLEL_ENABLE clause in a
//        SQL macro.
// ACTION:  Do not specify the PARTITION BY, CLUSTER BY, ORDER BY, DETERMINISTIC, AUTHID, or
//          PARALLEL_ENABLE clause with a SQL macro.
779, 0, "Formal parameters of type TABLE cannot have a default value."
// MANUAL: A default value cannot be specified for parameter of type TABLE.
// INDEX:  "SQL macros"
// RELEASE: 20.1
// CAUSE: A default value was specified for a formal parameter of type TABLE
//        in a SQL macro declaration.
// ACTION: Remove the default value from the parameter of type TABLE in
//         the SQL macro declaration.
//
780, 0, "Formal parameters of type COLUMNS may only have a NULL default value."
// MANUAL:  Parameters of type COLUMNS can only have a NULL default value.
// INDEX:  "SQL macros"
// RELEASE: 20.1
// CAUSE:  A non-NULL default value was specified for a formal parameter of type COLUMNS
//        in a polymorphic table function.
// ACTION: Change the default value for the parameter of type COLUMNS to
//         have a NULL default value in the polymorphic table function
//         specification.
//
//
781, 0, "SQL macro cannot be a type method"
// INDEX:  "SQL macros"
// RELEASE: 20.1
// CAUSE: The SQL macro could not be declared as a method in a type specification.
// ACTION: Use the SQL macro as a package function or a top level function.
//
782, 0, "A package containing SQL macro cannot be an invoker's rights package."
// INDEX:  "SQL macros"
// RELEASE: 20.1
// CAUSE: A package could not be declared as AUTHID CURRENT_USER if it had a SQL
//        macro function declared inside.
// ACTION: Declare the package as AUTHID DEFINER if it contains one or more
//         SQL macro functions.
Keep in mind that this functionality is not officially released and only a limited subset of it is available in 19.6: Bug 30324180 SQL Macro should be backported to 19.5. Hence, the final version of SQL Macro in Oracle 20 might be different from what we have now in Oracle 19.6.

понедельник, 20 января 2020 г.

Bug 30324180 SQL Macro should be backported to 19.5

When Oracle released 19.6 release update, I went to the usual My Oracle Support (MOS) note: Database 19 Release Updates and Revisions Bugs Fixed Lists (Doc ID 2523220.1) to find out what exactly was included there that might be of interest for me. Surprisingly, the note was not updated in several days, so that I decided to download the release update and take a look at it myself.
Oracle patches follow a well-defined structure that was described by Martin Berger: Oracle patches - a patchfiles anatomy.

One bug drew my eye:
<bug description="SQM MACRO SHOULD BE BACKPORTED TO 19.5" number="30324180"/>
The relevant note contains some important information, albeit without any examples:
Bug 30324180 - SQL Macro should be backported to 19.5 (Doc ID 30324180.8)
Thankfully, there is a clue that the table semantics macro should be supported, so that I decided to skim through an excellent Keith Laker's presentation on that subject: A New Approach to Simplifying Complex SQL-Using SQL Macros in Database 20c.

After a bit of toing and froing, I constructed the following example that demonstrates this new functionality (as it seems).
SQL> create table t1
  2  as
  3  select n,
  4         'val_'||to_char(n, 'fm00') val
  5    from xmltable('1 to 10'
  6           columns
  7             n int path '.');

Table created.

SQL>
SQL> create table t2
  2  as
  3  select 'val_'||to_char(n, 'fm00') s
  4    from xmltable('1 to 10'
  5           columns
  6             n int path '.');

Table created.

SQL>
SQL> select * from t1;

         N VAL
---------- -------
         1 val_01
         2 val_02
         3 val_03
         4 val_04
         5 val_05
         6 val_06
         7 val_07
         8 val_08
         9 val_09
        10 val_10

SQL> select * from t2;

S
-------
val_01
val_02
val_03
val_04
val_05
val_06
val_07
val_08
val_09
val_10
I created two tables with a different structure. The MOS note says about the table semantics macro, which is supposedly demonstrated on 22nd slide of Keith's presentation.
I was not able to get SQL_MACRO(TABLE) work, so that I ended up with the following example:
SQL> CREATE OR REPLACE FUNCTION sample(t DBMS_TF.Table_t, how_many number DEFAULT 5)
  2  RETURN VARCHAR2 SQL_MACRO
  3  AS
  4  BEGIN
  5    RETURN q'[SELECT *
  6                FROM t
  7               WHERE rownum <= how_many]';
  8  END sample;
  9  /

Function created.

SQL> select *
  2    from sample(t1, 3);

         N VAL
---------- -------
         1 val_01
         2 val_02
         3 val_03

SQL>
SQL> select *
  2    from sample(t1);

         N VAL
---------- -------
         1 val_01
         2 val_02
         3 val_03
         4 val_04
         5 val_05

SQL>
SQL> select *
  2    from sample(t2);

S
-------
val_01
val_02
val_03
val_04
val_05
I traced the last two statements through Intel Pin debugtrace and grepped the resulted file for the SQM string (which are presumably Oracle functions corresponding to the SQL Macro functionality):
<> qksptfSQM_Describe(0x7ff400bc6050, 0x6b76cbd8, ...)
| > qksptfSQM_Init(0x7ff400bc6050, 0x6b76cbd8, ...)
| | > qksptfSQM_Check_Errors(0x6b76cdf8, 0x7ff400b2c7a8, ...)
| | < qksptfSQM_Check_Errors+0x000000000090 returns: 0x7ff400b2c770
| < qksptfSQM_Init+0x0000000005d5 returns: 0x7ff400b2c7a8
| > qksptfSQM_GetTxt(0x7ff400b2c7a8, 0x1, ...)
| | | | | | | | | | | < qksptfSQM_GetTxt+0x000000001d51 returns: 0x6b76cdf8
| | | | | | | | | | | > qksptfSQM_Template(0x7ff400b2c7a8, 0, ...)
| | | | | | | | | | | < qksptfSQM_Template+0x0000000009ca returns: 0x6b76cdf8
| | | | | | | | | | | > qksptfSQM_Parse(0x7ff400b2c7a8, 0, ...)
| | | | | | | | | | | | | | | | | | > qksptfSQM_Parse_errors(0x7ff400b2c448, 0xc0, ...)
| | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_Parse_errors+0x00000000069a returns: 0
| | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_Model_Check(0x7ff400bba928, 0, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_Model_Check+0x000000000040 returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_Model_Check(0x7ff405adf108, 0, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_Model_Check+0x000000000040 returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_Model_Check(0x7ff405adde48, 0, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_Model_Check+0x000000000040 returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_Model_Check(0x7ff405ade8a8, 0, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_Model_Check+0x000000000040 returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_Model_Check(0x7ff400bc4260, 0, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_Model_Check+0x000000000040 returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_rm_vqb(0x7ff400bc4260, 0x7ff400bba928, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_rm_vqb+0x00000000003d returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_SetSQM_(0x7ff405adf108, 0x7ff400b2c7a8, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_SetSQM_(0x7ff400b2c110, 0x7ff400b2c7a8, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_SetSQM_+0x00000000007e returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_SetSQM_+0x00000000007e returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_SetSQM_(0x7ff405adde48, 0x7ff400b2c7a8, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_SetSQM_+0x00000000007e returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_SetSQM_(0x7ff405ade8a8, 0x7ff400b2c7a8, ...)
| | | | | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_SetSQM_+0x00000000007e returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_Parse+0x000000000851 returns: 0x1
| | | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_CleanupTDOs(0x3, 0x872027e0, ...)
| | | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_CleanupTDOs+0x0000000001cd returns: 0x7ff400a5d73c
| | | | | | | | | | | | > qksptfSQM_PstPrc(0x7ff400bba928, 0x98b7af08, ...)
| | | | | | | | | | | | | > qksptfSQM_PstPrc1_(0x7ff400bba928, 0x7ff400bba928, ...)
| | | | | | | | | | | | | < qksptfSQM_PstPrc1_+0x000000000032 returns: 0x1
| | | | | | | | | | | | | | > qksptfSQM_PstPrc1_(0x7ff405adf108, 0x7ff400bba928, ...)
| | | | | | | | | | | | | | | | > qksptfSQM_QbcRelExp_(0x7ff405adde48, 0x7ff405aded30, ...)
| | | | | | | | | | | | | | | | | | | > qksptfSQM_ExpRepl_(0x7ffef8fc52e0, 0x7ff405ad8818, ...)
| | | | | | | | | | | | | | | | | | | < qksptfSQM_ExpRepl_+0x00000000005a returns: 0
| | | | | | | | | | | | | | | | | | | > qksptfSQM_ExpRepl_(0x7ffef8fc52e0, 0x7ff405addf10, ...)
| | | | | | | | | | | | | | | | | | | < qksptfSQM_ExpRepl_+0x00000000005a returns: 0
| | | | | | | | | | | | | | | | | | | | > qksptfSQM_ExpRepl_(0x7ffef8fc52e0, 0x7ff405add890, ...)
| | | | | | | | | | | | | | | | | | | | < qksptfSQM_ExpRepl_+0x00000000005a returns: 0
| | | | | | | | | | | | | | | | | | | | | > qksptfSQM_ExpRepl_(0x7ffef8fc52e0, 0x7ff405add850, ...)
| | | | | | | | | | | | | | | | | | | | | < qksptfSQM_ExpRepl_+0x00000000005a returns: 0
| | | | | | | | | | | | | | | | | | | | | | > qksptfSQM_ExpRepl_(0x7ffef8fc52e0, 0x7ff405adda28, ...)
| | | | | | | | | | | | | | | | | | | | | | < qksptfSQM_ExpRepl_+0x00000000005a returns: 0
| | | | | | | | | | | | | | | | | | | | | > qksptfSQM_ExpRepl_(0x7ffef8fc52e0, 0x7ff405add858, ...)
| | | | | | | | | | | | | | | | | | | | | < qksptfSQM_ExpRepl_+0x00000000005a returns: 0
| | | | | | | | | | | | | | | | < qksptfSQM_QbcRelExp_+0x000000000079 returns: 0x1
| | | | | | | | | | | | | | | | | > qksptfSQM_QbcRelExp_(0x7ff400b2c110, 0x7ff405aded30, ...)
| | | | | | | | | | | | | | | | | | | | > qksptfSQM_ExpRepl_(0x7ffef8fc5270, 0x7ff405add1c8, ...)
| | | | | | | | | | | | | | | | | | | | < qksptfSQM_ExpRepl_+0x00000000005a returns: 0
| | | | | | | | | | | | | | | | | < qksptfSQM_QbcRelExp_+0x000000000079 returns: 0x1
| | | | | | | | | | | | | | < qksptfSQM_PstPrc1_+0x000000000181 returns: 0x1
| | | | | | | | | | | | | | | > qksptfSQM_PstPrc1_(0x7ff405adde48, 0x7ff400bba928, ...)
| | | | | | | | | | | | | | | < qksptfSQM_PstPrc1_+0x000000000032 returns: 0x1
| | | | | | | | | | | | | | | | > qksptfSQM_PstPrc1_(0x7ff400b2c110, 0x7ff400bba928, ...)
| | | | | | | | | | | | | | | | < qksptfSQM_PstPrc1_+0x000000000032 returns: 0x1
As Fritz Hoogland identified them, those are new functions introduced in Oracle 19.6: What’s new with Oracle database 19.6 versus 19.5. I will wait till Oracle officially announces SQL Macros. For those who cannot wait, they can play around with them even in 19.6. Judging by the bug description, the relevant functionality should be available for 19.5 (introduced without much fuss), but I have not found a one-off patch. It might have been set unpublished.