How to implement Data-Driven Apps – Using many Single Purpose Database or with a single Converged Database?

There is an on-going debate in our community about the best approach for developing cloud-native or data-driven apps. On one side, you have folks who say use a single-purpose “best-of-breed” database for each data type or workload you have. While the other half say, you should use a single converged database. So, which approach is right for you and your projects?

Let’s examine some of the pros and cons of each approach.

Single-purpose Databases

Single-purpose databases or purpose-built databases as they are often as known, are engineered to help solve a single or small number of problems. Given their narrow focus, they can ignore the tradeoffs usually required when trying to accommodate multiple data types or workloads. It also allows them to use a convenient data model that fits the purpose and to adopt APIs that seem natural for that data model. They offer less functionality than converged databases, and therefore, fewer APIs, making it easier to start developing against them. Their simplicity means they do a few things very well, but other things not at all. For example, a lot of single-purpose databases scale well, because they offer no strong consistency guarantees.

At first glance, single-purpose databases appear to be a good option. Developers are happy because they get exactly what they need to begin a project. However, when you look at the bigger picture, single-purpose databases can cause a lot of pain and end up costing more in the long run.

Very often, development requirements change mid-project, unforeseen business needs crop up, which render the original sound choice of single-purpose database “A”, sadly lacking. This leaves developers with a tough decision. Start from scratch with another single-purpose database to accommodate the new requirements and hoping that no others will surface, or work around the limitations of the original single-purpose database, adding unnecessary complexity to the application code and the maintenance of that code.

Organizations typically have a lot more than just one data type or workload to deal with. What happens when you have numerous single-purpose databases to support all of your business applications? What you quickly realize is that your data has become fragmented across different databases, which use different formats and types and have no direct way to integrate the data between them. How do you manage, secure or integrate such data with other parts of your organization?

Each single-purpose database is an entirely different database technology, with separate management controls, security models and high availability architectures. Sharing or propagating data across these databases is tricky because there isn’t a shared API.

Unfortunately, your organization must bear the brunt of the integration work required to make a single-purpose database approach feasible on a large scale. You will need personal who are knowledgeable about the operational aspects of each single-purpose database. Your security policies will have to re-implemented in every database, and your apps will become more complex to deal with propagating data from one database to another. Integrating single-purpose databases has the potential to become a job that never ends.

Converged Databases

Converged databases support all data types and workloads. They can also handle any development paradigm, including Microservices, Events, REST, and SaaS, to name just a few. By integrating data types, and workloads as features within a converged database, you can support mixed workloads and data types using a common language (SQL), and a standard set of APIs. You don’t need to manage and maintain multiple systems or worry about having to provide unified security across them.

By eliminating data fragmentation, you also eliminate copy contagion. Application modules or services automatically use a single copy of shared data in a converged database. There are no errors or time delays due to data propagated.

Application code overall becomes more straightforward, as you no longer have to work around the limitations of a single-purpose database or worry about integrating data from multiple sources or propagating data across numerous systems.

You also get synergy across datatypes and workloads. For example, by having support for Machine Learning algorithms and Spatial data in the same database, you can efficiently run predictive analytics on Spatial data. Data-driven app development becomes dramatically more straightforward and faster.

A converged database helps keep things less complicated, which in turn helps reduce the cost of implanting and maintaining your system. It also makes things more reliable.

In the video below, Juan Loaiza and Andrew Sutherland discuss this on-going debate and explain how a converged database makes it easy to develop Data-Driven Apps that enable enterprises to unlock endless possibilities and insights.

Single-purpose “best-of-breed” databases means no vendor lock-in, right?

Unfortunately, that’s not the case. Using a best-of-breed approach creates vendor lock-in, as each single-purpose database has proprietary APIs and transaction models instead of ISO standards like SQL. This fragments development and locks the application into the single-purpose database and very often, the particular cloud vendor providing that database.

It is dramatically simpler for developers to invoke extended SQL to execute ML, graph, spatial, blockchain, IoT and others in one converged database instead of implementing distributed execution and data movement across multiple databases.

But it’s expensive to get started with a Converged Database, right?

The introduction of database cloud services has removed the cost barrier to getting started with a converged database. Oracle Autonomous Database is an excellent example of a converged database. It’s available to developers as part of Oracle Cloud’s Free-Tier or and at a low hourly rate for production or pre-prod environments.

In the long run, organizations save time and money by using converged databases where they can manage all their data types and workloads with only one database technology that caters for all use cases and requirements from different parts of the organization.

Are there cases where a single-purpose database is the right choice?

As in other industries, there are new or boutique use cases that benefit from single-purpose or specialized products. Especially if the use-case exceeds the limits of a converged database in terms of cost, performance, or scalability, for example, a stock trading system or telephone exchange. These are latency-critical, transaction processing applications that require guaranteed microsecond response times and would benefit from being run on Oracle TimesTen.

The use of single-purpose databases should be limited to a small set of very specific tasks.

What are Query Block Names and how to find them

I got a lot of follow-up questions on what Query Block names are and how to find them, after my recent post about using SQL Patches to influence execution plans. Rather than burying my responses in the comment section under that post, I thought it would be more useful to do a quick post on it.

What are query blocks?

query block is a basic unit of SQL. For example, any inline view or subquery of a SQL statement are considered separate query blocks to the outer query.

The simple query below has just one sub-query, but it has two Query Blocks. One for the outer SELECT and one for the subquery SELECT.

Oracle automatically names each query block in a SQL statement based on the keyword using the following sort of name; sel$1, ins$2, upd$3, del$4, cri$5, mrg$6, set$7, misc$8, etc.

Given there are two SELECT statements in our query, the query block names will begin with SEL. The outer query will be SEL$1 and the inner query SEL$2.

How do I find the name of a query block?

To find the Query Block name, you can set the FORMAT parameter to ‘+alias’ in the DBMS_XPLAN.DISPLAY_CURSOR command. This will display the contents of the OBJECT_ALIAS column in the PLAN_TABLE, as a new section under the execution plan.

The new section will list the Query Block name for each of the lines in the plan.

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(format=>'+alias'));
 
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------
SQL_ID 4c8bfsduxhyht, child NUMBER 0
-------------------------------------
SELECT e.ename, e.deptno FROM emp e WHERE e.deptno IN (SELECT d.deptno 
FROM dept d WHERE d.loc='DALLAS')
Plan hash VALUE: 2484013818
---------------------------------------------------------------------------
| Id  | Operation	   | Name | ROWS  | Bytes | Cost (%CPU)| TIME	  |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |	  |	  |	  |	5 (100)|	  |
|*  1 |  HASH JOIN SEMI    |	  |	5 |   205 |	5  (20)| 00:00:01 |
|   2 |   TABLE ACCESS FULL| EMP  |    14 |   280 |	2   (0)| 00:00:01 |
|*  3 |   TABLE ACCESS FULL| DEPT |	1 |    21 |	2   (0)| 00:00:01 |
---------------------------------------------------------------------------
Query Block Name / Object Alias (IDENTIFIED BY operation id):
-------------------------------------------------------------
1 - SEL$5DA710D3
2 - SEL$5DA710D3 / E@SEL$1
3 - SEL$5DA710D3 / D@SEL$2
 
Predicate Information (IDENTIFIED BY operation id):
---------------------------------------------------
1 - access("E"."DEPTNO"="D"."DEPTNO")
3 - FILTER("D"."LOC"='DALLAS')

As you can see @SEL1 is the Query Block name for the outer query, where the EMP table is used and @SEL2 is the Query Block name for the sub-query, where the DEPT tables is used.

Continue reading “What are Query Block Names and how to find them”

What is a Converged Database?

At the recent OOW European conference there was a lot talk about Converged Databases and how they can greatly simplify data-driven app development.

But if you missed the conference, you might find yourself wondering what exactly is a Converged Database and what is the difference between a Converged Database and an Autonomous Database?

So, I thought it would be a good idea to write a short blog post explaining what a Converged Database is and how it relates to the Oracle Autonomous Database.

A Converged Database is a database that has native support for all modern data types (JSON, Spatial, Graph, etc. as well as relational), multiple workloads (IoT, Blockchain, Machine Learning, etc.) and the latest development paradigms (Microservice, Events, REST, SaaS, CI/CD, etc.) built into one product.

By having support for each of these datatype, workloads, and paradigms as features within a converged database, you can support mixed workloads and data types in a much simpler way. You don’t need to manage and maintain multiple systems or worry about having to provide unified security across them.

You also get synergy across these capabilities. For example, by having support for Machine Learning algorithms and Spatial data in the same database, you can easily do predictive analytics on Spatial data.  The Oracle Database is a great example of a Converged Database, as it provides support for Machine Learning, Blockchain, Graph, Spatial, JSON, REST, Events, Editions, and IoT Streaming as part of the core database at no additional cost.

A good analogy for a Converged Database is a smartphone. In the past, if you wanted to take a picture or video you would need a camera. If you wanted to navigate somewhere you would need a map or a navigation system. If you wanted to listen to music, you needed an iPod and if you wanted to make phone calls, you would also need a phone.

But with a smartphone, all of these products have been converged into one. Each of the original products is now a feature of the smartphone. Having all of these features converged into a single product inherently makes your life easier, as you can stream music over the phone’s data plan or upload pictures or videos directly to social media sites.
Continue reading “What is a Converged Database?”

SQL Tuning Workshop

Last week I had the pleasure of delivering a five-part SQL Tuning Workshop for my local Oracle User Group –  Northern California Oracle User Group. The workshop explains the fundamentals of the cost-based optimizer, the statistics that feed it, the hints that influence it and key tools you need to exam executions plans.

The workshop also provides a methodology for diagnosing and resolving the most common SQL execution performance problems. Given the volume of interest in this content, I want to share all of the material from the workshop here and give you links to additional material on each of the 5 topics.

Part 1 Understanding the Optimizer
The first part of the workshop covers the history of the Oracle Optimizer and explains the first thing the Optimizer does when it begins to optimize a query – query transformation.

Query transformations or the rewriting of the SQL statement into a semantically equivalent statement allows the Optimizer to consider alternative methods of processing or executing that query, which are often more efficient than the original SQL statement would allow. the majority of Oracle’s query transactions are now cost based, which means the Optimizer will cost the plan with and with the query transformation and pick the plan with the lowest cost. With the help of the Optimizer development team, I’ve already blogged about a number of these transformations including:

 

Part 2 Best Practices for Managing Optimizer Statistics
Part 2 of the workshop focuses on Optimizer Statistics and the best practices for managing them, including when and how to gather statistics, including fixed object statistics.
Continue reading “SQL Tuning Workshop”

Can I use an Autonomous Database to develop new applications?

Yes, Oracle Autonomous Database (ADB) is the ideal platform for new application development.

With this family of cloud services, developers no longer have to wait on others to provision hardware, install software, and create a database for them. With ADB, developers can easily and instantly deploy an Oracle database without worrying about having to manual tune it or capacity planning. This allows developers to start developing in minutes and concentrate on solving business problems without all of the usual distractions.

ADB has the most advanced SQL and PL/SQL support accelerating developer productivity by minimizing the amount of application code required to implement complex business logic. It also has a complete set of integrated Machine Learning algorithms, simplifying the development of applications that perform real-time predictions such as personalized shopping recommendations, customer churn rates, and fraud detection.

What Development Tools should I use with ATP?

Continue reading “Can I use an Autonomous Database to develop new applications?”

Oracle Database 19c is now available!

Today, April 25th, Oracle Database 19c became available to downloaded from Oracle.com.

Oracle Database 19c is the final member of the 12.2 family a.k.a 12.2.0.3 and is therefore the ‘long term support’ release. This means it will come with 4 years of premium support and 3 years of extended support. Making this release the version of the database that most folks are going to upgrade to next.

So, what can you expect?

There are hundreds of useful enhancements in Oracle Database 19c as well as a several new features. Dom Giles‘s latest post on the Oracle Database Insider blog has all the details on the new release, while I’ve listed just a couple of my personal favorites below.

Continue reading “Oracle Database 19c is now available!”