Part 4 of the “Who Owns the End-to-End Transaction?” 6-part blog series. Throughout the next few weeks, we will be taking a close look at how various IT stakeholders benefit from end-to-end transaction visibility. Be sure to check out:
Part 1: “Why Application Support Managers Need End-to-End Transaction Visibility”
Part 2: “IT Operations and End-to-End Visibility.”
Part 3: “Application Developers and End-to-End Transaction Visibility.”
Database Managers and the overall application performance management (APM) picture
Asking if the Database Manager (DBM) owns end-to-end transaction performance is obviously suspect. Who would possibly think that? No one is likely to hold the DBM accountable for the customer experience – at least not at first. But the DBM does play a critical role in the overall application performance management picture.
The Database Manager is responsible for how a company manages, organizes, stores, accesses and recovers its data. For most companies, data is the life-blood of its business. The end-to-end applications they deploy are, practically speaking, sophisticated tools for interacting with that data. Ease of access, integrity, security, and reliability of the data are what the DBM upholds. Their key responsibilities include:
- Modeling and designing databases, including defining data structure and relationships
- Implementing and deploying databases within data services
- Overseeing management of all the databases to ensure 24×7 availability
- Establishing and implementing policies and procedures to ensure the compliance, security and integrity of the data
- Optimizing stored procedures, indices, search and retrieval mechanisms
- Ensuring the performance of the data service supports the performance objectives of the end-to-end application
- Overseeing and managing all data extract/transform/load (ETL) migrations
- Documenting and maintaining application access interfaces
- Ensuring robust fail-over, backup and recovery practices in place to guarantee minimum down time
- Defining the means and standards for application developers to access the data service
In the last decade, the database and how its interacts with applications has evolved significantly. These changes have primarily been driven by Big Data and the elasticity and scalability advantages of cloud architectures which require highly distributed and scalable data access frameworks, and very complex and varied data structures. Increasing emphasis on business intelligence, data analytics, and real-time reporting has also meant that the database is under greater demand and dependency. In an effort to cope with the massive amounts of rich dynamic data, high levels of concurrency, and continuous data mining, innovative implementations such as Hadoop and the rise of noSQL databases have finally broken the classical database mold.
Managing the impact of database performance on the end-to-end system…and the end-customer experience
Today’s Database Manager still has a very tight focus on data and database services. But the path of data through the system can involve many complex hops and vary unpredictably. Different application components of the distributed system may use the database in different ways. Unintended interactions between the database and other application components may arise such as high levels of “chattiness”. These changes mean the DBM must now be more innovative when it comes to managing and measuring the accessibility and impact of the database on the entire system and the customer experience. To cope, the DBM needs to know much more about how the rest of the system is using the data – from individual instances of data access, to the aggregate rates of data exchange between services, to the relationship between the end-user experience and database performance.
With the Cloud comes new requirements
In the Cloud, where data becomes radically distributed, this level of visibility becomes invaluable. Application components can be unpredictably scaled across an enormous number of nodes which change constantly with load. Data access concurrency goes through the roof. Instances of failure or performance degradation become a nightmare to trace. These challenges give rise to new requirements:
- Performing continuous monitoring of the operational application to distinguish when and how it is being limited or impacted by the database access
- Measuring how DB response times contribute to the end-to-end response time
- Ensuring that data is being handled in accordance with compliance and data security requirements
- Troubleshooting specific instances of data access with respect to particular events elsewhere in the application, including customer experience
Network-based APM – Revealing the behavior of all applications in terms of database access and response times
Network-based Application Performance Management (APM) – otherwise known as Business Transaction Management (BTM) – offers the Database Manager end-to-end visibility into critical business transactions, monitors the performance of multi-tiered applications, and manages complex network infrastructures that involve a mixture of on-premise, virtual, Cloud-based, and third party software-as-a-service components. This allows the DBM to see not only transactions originating with the database, but all other transactions related to them. The behavior of the entire distributed application system can be revealed in terms of database access events or service transactions.
Transaction-level monitoring tools deliver a coherent, multidimensional set of views into the transactions at each level of the overall system that are shareable between the DBM and all the other teams who are invested in the application components. From that depth of information, the DBM can derive metrics that go far beyond simple access counts.
Using network-based APM, database transaction rates can be measured by request type, status, error codes, or even data content. They can be further qualified by the types and statuses of the other transactions that generated them. Statistics of DB transactions generated by specific type of transactions can be used to identify when and what parts of an application are too chatty. And DB response times and concurrency levels can be measured with respect to the end-user response time and to the response times of other involved services.
Network-based APM enables the DBM to assess the database performance and frames that performance in terms of the rest of the application. This gives the DBM the basis to push back to application developers when they implement database access poorly and to optimize database models to effectively accommodate the unexpected operational behavior of today’s complex and dynamic application.
Does the DBM really own the end-to-end transaction? Not so much – but being able to follow the movement of data throughout the distributed application allows the DBM to achieve optimal performance and work effectively with the teams who depend on it.
And now the DBM can own the influence of the database on the end-to-end performance of the most complex distributed applications.
For more information on network-based APM/BTM, check out our other blogs, whitepapers and videos on the Unified Transaction Model, and see the blogs in the rest of this series – Who Owns End-to-End Transaction?
You can also watch the on-demand webinar or download the whitepaper titled, “Who Owns the End-to-End Transaction: Mapping IT Stakeholders to Transaction-Derived Metrics”.