How to implement Data-Driven Apps – Using many Single Purpose Database or with a single Converged Database?

There is an on-going debate in our community about the best approach for developing cloud-native or data-driven apps. On one side, you have folks who say use a single-purpose “best-of-breed” database for each data type or workload you have. While the other half say, you should use a single converged database. So, which approach is right for you and your projects?

Let’s examine some of the pros and cons of each approach.

Single-purpose Databases

Single-purpose databases or purpose-built databases as they are often as known, are engineered to help solve a single or small number of problems. Given their narrow focus, they can ignore the tradeoffs usually required when trying to accommodate multiple data types or workloads. It also allows them to use a convenient data model that fits the purpose and to adopt APIs that seem natural for that data model. They offer less functionality than converged databases, and therefore, fewer APIs, making it easier to start developing against them. Their simplicity means they do a few things very well, but other things not at all. For example, a lot of single-purpose databases scale well, because they offer no strong consistency guarantees.

At first glance, single-purpose databases appear to be a good option. Developers are happy because they get exactly what they need to begin a project. However, when you look at the bigger picture, single-purpose databases can cause a lot of pain and end up costing more in the long run.

Very often, development requirements change mid-project, unforeseen business needs crop up, which render the original sound choice of single-purpose database “A”, sadly lacking. This leaves developers with a tough decision. Start from scratch with another single-purpose database to accommodate the new requirements and hoping that no others will surface, or work around the limitations of the original single-purpose database, adding unnecessary complexity to the application code and the maintenance of that code.

Organizations typically have a lot more than just one data type or workload to deal with. What happens when you have numerous single-purpose databases to support all of your business applications? What you quickly realize is that your data has become fragmented across different databases, which use different formats and types and have no direct way to integrate the data between them. How do you manage, secure or integrate such data with other parts of your organization?

Each single-purpose database is an entirely different database technology, with separate management controls, security models and high availability architectures. Sharing or propagating data across these databases is tricky because there isn’t a shared API.

Unfortunately, your organization must bear the brunt of the integration work required to make a single-purpose database approach feasible on a large scale. You will need personal who are knowledgeable about the operational aspects of each single-purpose database. Your security policies will have to re-implemented in every database, and your apps will become more complex to deal with propagating data from one database to another. Integrating single-purpose databases has the potential to become a job that never ends.

Converged Databases

Converged databases support all data types and workloads. They can also handle any development paradigm, including Microservices, Events, REST, and SaaS, to name just a few. By integrating data types, and workloads as features within a converged database, you can support mixed workloads and data types using a common language (SQL), and a standard set of APIs. You don’t need to manage and maintain multiple systems or worry about having to provide unified security across them.

By eliminating data fragmentation, you also eliminate copy contagion. Application modules or services automatically use a single copy of shared data in a converged database. There are no errors or time delays due to data propagated.

Application code overall becomes more straightforward, as you no longer have to work around the limitations of a single-purpose database or worry about integrating data from multiple sources or propagating data across numerous systems.

You also get synergy across datatypes and workloads. For example, by having support for Machine Learning algorithms and Spatial data in the same database, you can efficiently run predictive analytics on Spatial data. Data-driven app development becomes dramatically more straightforward and faster.

A converged database helps keep things less complicated, which in turn helps reduce the cost of implanting and maintaining your system. It also makes things more reliable.

In the video below, Juan Loaiza and Andrew Sutherland discuss this on-going debate and explain how a converged database makes it easy to develop Data-Driven Apps that enable enterprises to unlock endless possibilities and insights.

Single-purpose “best-of-breed” databases mean no vendor lock-in, right?

Unfortunately, that’s not the case. Using a best-of-breed approach creates vendor lock-in, as each single-purpose database has proprietary APIs and transaction models instead of ISO standards like SQL. This fragments development and locks the application into the single-purpose database and very often, the particular cloud vendor providing that database.

It is dramatically simpler for developers to invoke extended SQL to execute ML, graph, spatial, blockchain, IoT and others in one converged database instead of implementing distributed execution and data movement across multiple databases.

But it’s expensive to get started with a Converged Database, right?

The introduction of database cloud services has removed the cost barrier to getting started with a converged database. Oracle Autonomous Database is an excellent example of a converged database. It’s available to developers as part of Oracle Cloud’s Free-Tier or and at a low hourly rate for production or pre-prod environments.

In the long run, organizations save time and money by using converged databases where they can manage all their data types and workloads with only one database technology that caters for all use cases and requirements from different parts of the organization.

Are there cases where a single-purpose database is the right choice?

As in other industries, there are new or boutique use cases that benefit from single-purpose or specialized products. Especially if the use-case exceeds the limits of a converged database in terms of cost, performance, or scalability, for example, a stock trading system or telephone exchange. These are latency-critical, transaction processing applications that require guaranteed microsecond response times and would benefit from being run on Oracle TimesTen.

The use of single-purpose databases should be limited to a small set of very specific tasks.

Just a quick note of thanks to Gerald Venzl for keeping me honest on the single-purpose database approach!

Generating an AWR report for Autonomous Databases

Oracle Autonomous Database automates the lifecycle management of a database, everything from provisioning, scaling, backups and patching, but what it doesn’t do yet is fully tune your application.

You are still on the hook to make sure your app doesn’t have any concurrency bottlenecks or poorly written SQL that Auto Indexing can’t address.

So, what can you do to monitor your app while it’s running on an Autonomous Database?

The first place you can start is the Performance Hub tab on the cloud console. Here you’ll find both real-time and historical performance data in the form of Active Session History (ASH) information for the last seven days, and SQL Monitor reports for the high-load SQL. You can aggregate the ASH data several different ways including by wait class, database service, resource group or SQL ID. The same information is also available in Oracle Management Cloud (OMC), and SQL Developer Web.

But if you are trying to get a holistic view of how your app is behaving on a database, nothing beats an Automatic Workload Repository report or AWR report.

Continue reading “Generating an AWR report for Autonomous Databases”

What are Query Block Names and how to find them

I got a lot of follow-up questions on what Query Block names are and how to find them, after my recent post about using SQL Patches to influence execution plans. Rather than burying my responses in the comment section under that post, I thought it would be more useful to do a quick post on it.

What are query blocks?

query block is a basic unit of SQL. For example, any inline view or subquery of a SQL statement are considered separate query blocks to the outer query.

The simple query below has just one sub-query, but it has two Query Blocks. One for the outer SELECT and one for the subquery SELECT.

Oracle automatically names each query block in a SQL statement based on the keyword using the following sort of name; sel$1, ins$2, upd$3, del$4, cri$5, mrg$6, set$7, misc$8, etc.

Given there are two SELECT statements in our query, the query block names will begin with SEL. The outer query will be SEL$1 and the inner query SEL$2.

How do I find the name of a query block?

To find the Query Block name, you can set the FORMAT parameter to ‘+alias’ in the DBMS_XPLAN.DISPLAY_CURSOR command. This will display the contents of the OBJECT_ALIAS column in the PLAN_TABLE, as a new section under the execution plan.

The new section will list the Query Block name for each of the lines in the plan.

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(format=>'+alias'));
 
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------
SQL_ID 4c8bfsduxhyht, child NUMBER 0
-------------------------------------
SELECT e.ename, e.deptno FROM emp e WHERE e.deptno IN (SELECT d.deptno 
FROM dept d WHERE d.loc='DALLAS')
Plan hash VALUE: 2484013818
---------------------------------------------------------------------------
| Id  | Operation	   | Name | ROWS  | Bytes | Cost (%CPU)| TIME	  |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |	  |	  |	  |	5 (100)|	  |
|*  1 |  HASH JOIN SEMI    |	  |	5 |   205 |	5  (20)| 00:00:01 |
|   2 |   TABLE ACCESS FULL| EMP  |    14 |   280 |	2   (0)| 00:00:01 |
|*  3 |   TABLE ACCESS FULL| DEPT |	1 |    21 |	2   (0)| 00:00:01 |
---------------------------------------------------------------------------
Query Block Name / Object Alias (IDENTIFIED BY operation id):
-------------------------------------------------------------
1 - SEL$5DA710D3
2 - SEL$5DA710D3 / E@SEL$1
3 - SEL$5DA710D3 / D@SEL$2
 
Predicate Information (IDENTIFIED BY operation id):
---------------------------------------------------
1 - access("E"."DEPTNO"="D"."DEPTNO")
3 - FILTER("D"."LOC"='DALLAS')

As you can see @SEL1 is the Query Block name for the outer query, where the EMP table is used and @SEL2 is the Query Block name for the sub-query, where the DEPT tables is used.

Continue reading “What are Query Block Names and how to find them”

What is a Converged Database?

At the recent OOW European conference there was a lot talk about Converged Databases and how they can greatly simplify data-driven app development.

But if you missed the conference, you might find yourself wondering what exactly is a Converged Database and what is the difference between a Converged Database and an Autonomous Database?

So, I thought it would be a good idea to write a short blog post explaining what a Converged Database is and how it relates to the Oracle Autonomous Database.

What is a Converged Database?

A Converged Database is a database that has native support for all modern data types (JSON, Spatial, Graph, etc. as well as relational), multiple workloads (IoT, Blockchain, Machine Learning, etc.) and the latest development paradigms (Microservice, Events, REST, SaaS, CI/CD, etc.) built into one product.

By having support for each of these datatype, workloads, and paradigms as features within a converged database, you can support mixed workloads and data types in a much simpler way. You don’t need to manage and maintain multiple systems or worry about having to provide unified security across them.

You also get synergy across these capabilities. For example, by having support for Machine Learning algorithms and Spatial data in the same database, you can easily do predictive analytics on Spatial data.  The Oracle Database is a great example of a Converged Database, as it provides support for Machine Learning, Blockchain, Graph, Spatial, JSON, REST, Events, Editions, and IoT Streaming as part of the core database at no additional cost.

A good analogy for a Converged Database is a smartphone. In the past, if you wanted to take a picture or video you would need a camera. If you wanted to navigate somewhere you would need a map or a navigation system. If you wanted to listen to music, you needed an iPod and if you wanted to make phone calls, you would also need a phone.

But with a smartphone, all of these products have been converged into one. Each of the original products is now a feature of the smartphone. Having all of these features converged into a single product inherently makes your life easier, as you can stream music over the phone’s data plan or upload pictures or videos directly to social media sites.
Continue reading “What is a Converged Database?”

How to use a SQL Plan Baseline or a SQL Patch to add Optimizer hints

In a recent chat with Connor McDonald, we discussed if it is realistic to have applications that don’t contain Optimizer hints. Ideally, the answer to this question is “yes”, you don’t need hints if you have a well-written application and you have supplied a representative set of statistics and all the possible constraint information (NOT NULL, Primary keys, Foreign Keys, etc.) to the Optimizer.

But in reality even with all of this in place, there can be cases where something goes wrong with the execution plan for a critical SQL statement and you get called in to fix.

During our chat, Connor used a very apt analogy to describe this situation. He said it was like having a patient arrive in the emergency room, who is bleeding profusely. Your first priority is to stop the patient from bleeding by slapping a band-aid on the wound.

The same is true for our poorly performing SQL statement. Our initial response is to add an optimizer hint to get the SQL statement’s execution plan back to a reasonable response time or acceptable performance.

But once a patient has been stabilized in the emergency room, medical professionals normally take that patient into surgery to make a permanent fix or at the very least stitch up the wound properly.

We need to make sure we do the same thing for our SQL statements.

Rather than leaving a band-aid in the application code in the form of an optimizer hint, we should either fix the root cause or at the very least, make a permanent fix that can be easily traced and ideally can evolve over time.

That’s why you often hear me say, “if you can hint it, you can baseline or patch it”.

What do I mean by that?

I mean we should capture the hinted plan as a SQL plan baseline or at the very least insert the hints via a SQL Patch so that we know that this statement is patched (the use of a SQL patch is visible in the note section of the plan).

Continue reading “How to use a SQL Plan Baseline or a SQL Patch to add Optimizer hints”

SQL Tuning Workshop

Last week I had the pleasure of delivering a five-part SQL Tuning Workshop for my local Oracle User Group –  Northern California Oracle User Group. The workshop explains the fundamentals of the cost-based optimizer, the statistics that feed it, the hints that influence it and key tools you need to exam executions plans.

The workshop also provides a methodology for diagnosing and resolving the most common SQL execution performance problems. Given the volume of interest in this content, I want to share all of the material from the workshop here and give you links to additional material on each of the 5 topics.

Part 1 Understanding the Optimizer
The first part of the workshop covers the history of the Oracle Optimizer and explains the first thing the Optimizer does when it begins to optimize a query – query transformation.

Query transformations or the rewriting of the SQL statement into a semantically equivalent statement allows the Optimizer to consider alternative methods of processing or executing that query, which are often more efficient than the original SQL statement would allow. the majority of Oracle’s query transactions are now cost based, which means the Optimizer will cost the plan with and with the query transformation and pick the plan with the lowest cost. With the help of the Optimizer development team, I’ve already blogged about a number of these transformations including:

 

Part 2 Best Practices for Managing Optimizer Statistics
Part 2 of the workshop focuses on Optimizer Statistics and the best practices for managing them, including when and how to gather statistics, including fixed object statistics.
Continue reading “SQL Tuning Workshop”

How do I scale an Autonomous Database?

Traditionally database deployments have been designed and provision for the peak possible workload. And in reality, a substantial margin of safety was also provisioned on top of that in order to make sure the system could cope with any unforeseen demands.
But peak workloads tend to occur infrequently, leaving most of this costly capacity idle the majority of the time.

In order to enable customers to pay for only the resources they need, Oracle Autonomous Database allow customers to elastically adjust their compute and storage resources when necessary.

An Autonomous Database can be scaled through the UI as shown in the video below or via our cloud APIs or CLI commands. In the video you see how I can scale an 2 CPU  configuration to an 8 CPU configuration in under a minute  to accommodate 48 concurrent users running a JSON workload.

Continue reading “How do I scale an Autonomous Database?”

OpenWorld 2019 Wrap Up

I can’t believe Oracle Open World is done for another year.

But what a great conference it was for those interested in the Oracle Database. We saw the introduction of Oracle Database 20c and of course the continuing evolution of the Oracle Autonomous Database with details on new deployment option (Dedicated) and new flavors (Document Store).

If you weren’t able to make it to OOW in person, don’t panic as a lot of the session presentations are now available for download in the OOW session catalog. You can download the slides by clicking the down arrow on the right-hand side of each session title as shown below.

Continue reading “OpenWorld 2019 Wrap Up”

FREE Oracle Autonomous Database

Today was the first day of Oracle Open World and during Larry Ellison’s afternoon keynote he made a number of extremely exciting announcements around Oracle Autonomous Database including the launch of an Always Free Tier for Developers and students in the Oracle Cloud.  Larry said, “as long as you use the service, it will be there, free forever.”

So what’s included in Oracle’s Always Free Tier?

Along with compute and storage services, you also get access to 2 Autonomous Database with 20GB of storage each absolutely free. So you can try out both an Autonomous Data Warehouse and an Autonomous Transaction Processing system.

Continue reading “FREE Oracle Autonomous Database”

Are you ready for Oracle OpenWorld 2019?

With less than a week to go until Oracle Open World kicks off, I thought I would share with you what you can expect.

Of course the whole Database team will be there and you will have multiple opportunities to meet up with us, in one of our technical sessions, our hands-on-labs or at the Oracle demogrounds.

Below are just some of the session I thought might be interesting to help get you started:

Continue reading “Are you ready for Oracle OpenWorld 2019?”