In 11g, Oracle introduced the DBMS_STAT.DIFF_TABLE_STATS functions to help you compare two sets of statistics for a table along with all its dependent objects (indexes, columns, partitions).
There are three versions of this function depending on where the statistics being compared are located:
DBMS_STAT.DIFF_TABLE_STATS_IN_HISTORY (compares statistics for a table from two timestamps in the past)
DBMS_STAT.DIFF_TABLE_STATS_IN_PENDING (compares pending statistics and statistics as of a timestamp or statistics from the data dictionary)
DBMS_STAT.DIFF_TABLE_STATS_IN_STATTAB (compares statistics from a user statistics table and the data dictionary, from two different user statistics tables, or a single user statistics table using two different STATSIDs)
The functions return a report that has three sections:
Basic table statistics The report compares the basic table statistics (number of rows, blocks, etc.).
Column statistics The second section of the report examines column statistics, including histograms.
Index Statistics The final section of the report covers differences in index statistics.
Statistics will only be displayed in the report if the difference in the statistics exceeds a certain threshold (%). The threshold can be specified as an argument to the functions (PCTTHRESHOLD); the default value is 10%. The statistics corresponding to the first source, typically the current table stats in the data dictionary, will be used to compute the differential percentage.
The functions also return theMAXDIFFPCT(a number) along with the report. This is the maximum percentage difference between the statistics. These differences can come from the table, column, or index statistics.
Partitioning is one of the most powerful tools at your disposal when managing large volumes of data and improving the performance of queries that would otherwise scan and filter lots and lots of data.
However, it can be tricky to determine why it isn’t helping when you thought it should.
Typically partitioning improves query performance by ensuring only the partition(s) needed to answer the business user’s query will be scanned rather than the entire table.
But how can you tell if you got partition pruning in an execution plan? Or worse yet, how do you determine why you didn’t get partitioning pruning when you were expecting it.
In the video below, I share the steps I use to determine if partition pruning has occurred and what to look at and correct if you don’t automatically get the partition pruning you were expecting.
This blog post is part of a series on SQL Tuning. I shared some simple steps to help you tune a SQL Statement using the wrong Join Type in part one. While part two deals with how to tackle a problem where the optimizer picks the wrong index, and part three shares tips on how to fix statements where the Optimizer chooses a nested loop join instead of a hash join.
When it comes to SQL Tuning, I don’t typically recommend folks add one-off hints or look for magical underscore parameters to help improve their query performance.
Instead, I like to share some simple steps to
Accurately characterise the problem
Determine and apply a solution
Measure the effectiveness of that solution
But instead of talking about my approach, I thought it would be more beneficial to show you how I do it. So in the video below, you will see the exact steps I used to determine what was causing a Sales Report to run slowly and the two alternative approaches you can take to resolve similar problems.
Continuing my blog series on reading and interpreting Oracle execution plans, this week’s post covers Join Order.
So what is the Join Order?
The join order is the order in which the tables are joined together in a multi-table SQL statement. Ideally, a plan should start with the join that eliminates the most data to minimize the amount of data carried into the subsequent joins.
How is the Join Order Determined?
The join order is determined based on cost, which is strongly influenced by the cardinality estimates and available access paths. However, the Optimizer will also always adhere to some basic rules:
The Optimizer will always select a join that will produce at most 1 row as the initial join in the plan. For example, a join between two row sources that only have 1 row each. Like a primary key lookup or an index unique scan.
If a SQL statement uses an outer join, then the optimizer must obey the join order specified by the outer join. That is to say; the row preserving table must come after the other table in the predicate to ensure all of the additional rows that don’t satisfy the join condition can be added to the result set correctly. For example, with the Oracle syntax for outer joins, the table with the outer join operator must come after the other table in the predicate. Thus, in this example, the cites table must come before the countries table.
WHERE cities.country_id = countries.id(+);
For SQL statements that reference a database view, the Optimizer will attempt to do view merging, where the definition of the view is inserted into the rest of the SQL statement, and the entire expanded statement is optimized as a whole. However, there are a few cases where view merging isn’t possible. In these cases, the optimizer will join all of the tables in the view together before the resulting data set is joined to the tables outside the view.
When a subquery has been converted into an anti-join (NOT IN subquery) or semi-join (EXISTS subquery), the tables from the subquery must come after those tables in the outer query block to which they were connected or correlated. However, hash anti-joins and semi-joins can override this ordering condition under certain circumstances.
How to determine the Join Order in an execution plan
You can take several approaches to determine the Join Order in a plan, from looking at the indentation of the tables in the operation column to a depth-first search. To clearly explain how to identify the Join Order in an execution plan, I’ve created a short video demonstrating several approaches using real-world examples.
What if I don’t get the Join Order I want?
The leading cause of the wrong Join Order is typically a cardinality misestimate on the table or joins in the query or missing access methods.
Continuing my blog series on reading and interpreting Oracle execution plans, this week’s post covers the different Join Methods and types available to the Optimizer.
What is an Oracle Join Method?
Join Methods are the techniques used by the Oracle Optimizer to join data coming from two data producing operators in an execution plan. You can find the individual Join Methods chosen by the Optimizer in the Operations column of an Execution table.
How many Join Methods does the Optimizer have to choose from?
The Oracles Optimizer supports three join methods; Nested Loops, Hash Join and Sort Merge Join.
Nested Loops Join
Nested loops joins are useful when small subsets
of data are being joined and if there is an efficient way of accessing the second table (for example an index lookup).
For every row in the first table (the outer table), Oracle accesses all the rows in the second table (the inner table) looking for a match. You can think of it as a set of embedded FOR loops.
NOTE: In Oracle Database 11g the internal implementation for nested loop joins changed to reduce overall latency for physical I/O. You may see two NESTED LOOPS operators in the plan’s operations column, where you previously only saw one on earlier versions of Oracle. You can find more details on why there are two operators in the video below.
Hash Joins
Hash joins are used for joining large data sets. The Optimizer uses the smaller of the two tables or data sources to build a hash table, based on the join key, in memory. It then scans the larger table and performs the same hashing algorithm on the join column(s). It then probes the previously built hash table for each value and if they match, it returns a row.
Sort Merge Joins
Sort Merge joins are useful when the join condition between two tables is an in-equality condition such as, <, <=, >, or >=. Sort merge joins can perform better than nested loop joins for large data sets. The join consists of two steps:
Sort join operation: Both the inputs are sorted on the join key.
Merge join operation: The sorted lists are merged together.
A Sort Merge join is more likely to be chosen if there is an index on one of the tables that will eliminate one of the sorts.
When will the Optimizer choose each of these methods, and what can I do to influence that decision?
To clearly explain how each of the Join Methods works and when they will be chosen, I’ve created the short video below.
What if I don’t get the Join Method I want?
The leading cause of getting the wrong Join Method is typically a cardinality misestimate on the table on the left-hand side of the join. That’s why Oracle introduced Adaptive Plans and more specifically Adaptive Join Methods in Oracle Database 12c to help automatically correct itself if the wrong Join Method is chosen.
How Adaptive Joins work
During the initial execution of a plan, if Oracle detects that the Optimizer’s cardinality estimates were wrong, the join method can be changed “on the fly” to a better option.
It’s possible to change an adaptive plan “on the fly” because it consists of a default plan, which is the plan that the Optimizer picks based on the current statistics and an alternative method for various portions of the plan. For example, the default plan could be a Nested Loops plan, and the alternative(subplan) would be a Hash join.
A new operated called a Statistics Collector is inserted into the plan, right above the table on the left-hand side of the join, which will buffer the rows coming out of the table until we can get a sense of how many rows will be returned. Once we know the number of rows returned or the number is above a certain threshold, the Optimizer will choose the final join method. After the initial execution, the Statistics Collector and the subplan components not chosen become no-ops, and the impact on execution plan performance is nill.
At the end of last year, I began a blog series on reading and interpreting Oracle execution plans. In this week’s post, I will tackle the aspect of execution plans that I get the most questions about, Access Methods.
What are Oracle Access Paths or Methods?
Access Methods or Access Paths are the techniques used by Oracle to access the data needed to answer a query. Access Methods can be divided into two categories; data accessed via a table scan or index access. You can find the individual Access Methods chosen by the Optimizer in the Operations column of an Execution table.
How Many Access Paths are available to the Optimizer?
Oracle supports nine different Access Methods today, as shown in the image below.
When will the Optimizer choose each of these methods, and what can I do to influence that decision?
To clearly explain how each of the Access Methods works and when it will be chosen, I’ve created a short video.
What if I don’t get the Access Method I want?
If the Access Method you see in an execution plan is not what you expect, check the cardinality estimates for that object are correct, and the join order allows the access method you desire. Remember, Optimizer transformations (the rewriting of your query to open up additional access methods) can also greatly impact the Access Method.
In last week’s post, I began a series on how to read and interpret Oracle execution plans by explaining what an execution plan is and how to generate one. This week I’m going to tackle the most important piece of information the Optimizer shares with you via the execution plan, it’s cardinality estimates.
What is a Cardinality Estimate?
A cardinality estimate is the estimated number of rows, the optimizer believes will be returned by a specific operation in the execution plan. The Optimizer determines the cardinality for each operation based on a complex set of formulas that use table and column level statistics as input (or the statistics derived by dynamic sampling). It’s considered the most important aspect of an execution plan because it strongly influences all of the other decisions the optimizer makes.
In part 4 of our series, I share some of the formulas used by the optimizer to estimate cardinalities, as well as showing you how to identify cardinalities in a plan. I also demonstrate multiple ways to determine if the cardinality estimates are accurate.
What can cause a Cardinality Misestimate and how do I fix it?
Several factors can lead to incorrect cardinality estimates even when the basic table and column statistics are up to date. In part 5 of our series, I explain the leading causes of cardinality misestimates and how you can address them.
Next weeks, instalment will be all about the different access methods available to the Optimizer and what you can do to encourage the optimizer to select the access method you want!
Examining the different aspects of an execution plan, from cardinality estimates to parallel execution, and understanding what information you should glean from it can be overwhelming even for the most experienced DBA.
That’s why I’ve put together a series of short videos that will walk you through each aspect of the plan and explain what information you can find there and what to do if the plan isn’t what you were expecting.
What is an Execution Plan?
The series starts at the very beginning with a comprehensive overview of what an execution plan is and what information is displayed in each section. After all, you can’t learn to interpret what is happening in a plan, until you know what a plan actually is.
How to Generate an Execution Plan?
Although multiple different tools will display an Oracle Execution Plan for you, there really are only two ways to generate the plan. You can use the Explain Plan command, or you can view the execution plan of a SQL statement currently in the Cursor Cache using the dictionary view V$SQL_Plan. This session covers both techniques for you and provides insights into what additional information you can get the Optimizer to share with you when you generate a plan. It also explains why you don’t always get the same plan with each approach, as I discussed in an earlier post.
How to use DBMS_XPLAN to FORMAT an Execution Plan
The FORMAT parameter within the DBMS_XPLAN.DISPLAY_CURSOR function is the best tool to show you detailed information about a what’s happened in an execution plan including the bind variable values used, the actual number of rows returned by each step, and how much time was spent on each step. I’ve also covered a lot of the content in this video in a previous post.
Part 2 of the series covers Cardinality Estimates and what you can do to improve them!
Part 3 of the series covers access Methods and what you can do if you don’t get the access method you were expecting.
Part 4 of the series covers Join Methods and when you can expect each one and what to do if you don’t get the join method you were expecting.
Remember you can always get more information on the Oracle Optimizer on the Optimizer team’s blog.
Teaches you to apply the techniques discussed in the previous sections to help diagnose and correct a number of problematic SQL statements.
The final part of the SQL Tuning workshop focuses on applying the techniques discussed in the previous sections to help diagnose and correct a number of problematic SQL statements and shows how you can use SQL Plan Management or a SQL Patch to influence an execution plan.
From time to time, it may become necessary to influence the plan the Optimizer chooses via Optimizer hints. This session explains how Optimizer hints are interpreted, when and where they should be used.
Part 4 is called “Harnessing the power of optimizer hints”. Although I am not a strong supporter of adding hints to SQL statements for a whole host of reasons, from time to time, it may become necessary to influence the plan the Optimizer chooses. The most powerful way to alter the plan chosen is via Optimizer hints. But knowing when and how to use Optimizer hints correctly is somewhat of a dark art. This session explains how Optimizer hints are interpreted, when and where they should be used, and why they sometimes appear to be ignored.
