Auto Tuning Index Azure Database Report Example
Alter database current set automatictuning (forcelastgoodplan = on, createindex = default, dropindex = off) The forcelastgoodplan flag will work in SQL Server 2017 as well. This is what recommendations look like in Azure (I took an image from MSDN as I didn’t have any good examples to show you from my own environment).
- Auto Tuning Index Azure Database Report Example Free
- Auto Tuning Index Azure Database Report Example Pdf
- Auto Tuning Index Azure Database Report Example In Excel
May 16, 2016 Simplify performance tuning and troubleshooting with Azure SQL Database. Posted on 16 May, 2016. Overview of the current state of Auto-tuning features. Azure SQL Database makes performance tuning and troubleshooting easier and faster than ever before, helping you to deliver great performance for your database applications while saving. Configure automatic tuning on an individual database only if you need that database to have different settings than others inheriting settings from the same server. Azure portal. To enable automatic tuning on a single database, navigate to the database in Azure portal and select Automatic tuning. May 16, 2016 Automated Performance Tuning. Tired of managing the indexes for your databases? Well you don’t have to do this yourself anymore - you can now tell Azure SQL Database that you’d like to automate the implementation of certain types of recommendations. It is now possible to set automatic tuning option through ARM template at the logical server or the database level. I've used the Automation Script blade on the Azure portal to get these information: Sql server level.
Automatic tuning is a database feature that provides insight into potential query performance problems, recommend solutions, and automatically fix identified problems.
Automatic tuning in SQL Server 2017 (14.x) notifies you whenever a potential performance issue is detected, and lets you apply corrective actions,or lets the Database Engine automatically fix performance problems.Automatic tuning in SQL Server 2017 (14.x) enables you to identify and fix performance issues caused by query execution plan choice regressions. Automatic tuning in Azure SQL Database also creates necessary indexes and drops unused indexes. For more information on query execution plans, see Execution Plans.
The SQL Server Database Engine monitors the queries that are executed on the database and automatically improves performance of the workload. The Database Engine has a built-in intelligence mechanism that can automatically tune and improve performance of your queries by dynamically adapting the database to your workload. There are two automatic tuning features that are available:
- Automatic plan correction identifies problematic query execution plans and fixes query execution plan performance problems. Applies to: SQL Server (Starting with SQL Server 2017 (14.x)) and Azure SQL Database
- Automatic index management identifies indexes that should be added in your database, and indexes that should be removed. Applies to: Azure SQL Database
Why automatic tuning?
Three of the main tasks in classic database administration are monitoring the workload, identifying critical Transact-SQL queries, indexes that should be added to improve performance, and identifying rarely used. The SQL Server Database Engine provides detailed insight into the queries and indexes that you need to monitor. However, constantly monitoring a database is a hard and tedious task, especially when dealing with many databases. Managing a huge number of databases might be impossible to do efficiently. Instead of monitoring and tuning your database manually, you might consider delegating some of the monitoring and tuning actions to the Database Engine using automatic tuning feature.
How does automatic tuning work?
Automatic tuning is a continuous monitoring and analysis process that constantly learns about the characteristics of your workload and identify potential issues and improvements.
This process enables database to dynamically adapt to your workload by finding what indexes and plans might improve performance of your workloads and what indexes affect your workloads. Based on these findings, automatic tuning applies tuning actions that improve performance of your workload. In addition, database continuously monitors performance after any change made by automatic tuning to ensure that it improves performance of your workload. Any action that didn't improve performance is automatically reverted. This verification process is a key feature that ensures that any change made by automatic tuning does not decrease the performance of your workload.
Automatic plan correction
Automatic plan correction is an automatic tuning feature that identifies execution plans choice regression and automatically fix the issue by forcing the last known good plan. For more information about query execution plans and the query optimizer, see the Query Processing Architecture Guide.
What is execution plan choice regression?
The SQL Server Database Engine may use different execution plans to execute the Transact-SQL queries. Query plansdepend on the statistics, indexes, and other factors. The optimal plan that should be used to execute some Transact-SQL query might be changedover time. In some cases, the new plan might not be better than the previous one, and the new plan might cause a performance regression.
Whenever you notice the plan choice regression, you should find some previous good plan and force it instead of the current one using sp_query_store_force_plan
procedure.Database Engine in SQL Server 2017 (14.x) provides information about regressed plans and recommended corrective actions.Additionally, Database Engine enables you to fully automate this process and let Database Engine fix any problem found relatedto the plan changes.
Automatic plan choice correction
Database Engine can automatically switch to the last known good plan whenever the plan choice regression is detected.
Database Engine automatically detects any potential plan choice regression including the plan that should be used instead of the wrong plan.When the Database Engine applies the last known good plan, it automatically monitors the performance of the forced plan. If the forced plan is not betterthan the regressed plan, the new plan will be unforced and the Database Engine will compile a new plan. If the Database Engine verifies that the forced plan is better than the regressed plan, the forced plan will be retained if it is better than the regressed plan, until a recompile occurs (for example, on next statistics update or schema change).
Note
Any execution plans auto forced are not persisted between restarts of the SQL Server instance.
Enabling automatic plan choice correction
You can enable automatic tuning per database and specify that last good plan should be forced whenever some plan change regression is detected. Automatic tuning is enabled usingthe following command:
Once you enable this option, the Database Engine will automatically force any recommendation where the estimated CPU gain is higher than 10 seconds, or the number of errors in the new plan is higher than the number of errors in the recommended plan, and verify that the forced plan is better than the current one.
Alternative - manual plan choice correction
Without automatic tuning, users must periodically monitor system and look for the queries that regressed. If any plan regressed, user should find some previous good plan and force it instead of the current one using sp_query_store_force_plan
procedure. The best practice would be to force the last known good plan because older plans might be invalid due to statistic or index changes. The user who forces the last known good plan should monitor performance of the query that is executed using the forced plan and verify that forced plan works as expected. Depending on the results of monitoring and analysis, plan should be forced or user should find some other way to optimize the query.Manually forced plans should not be forced forever, because the Database Engine should be able to apply optimal plans. The user or DBA should eventuallyunforce the plan using sp_query_store_unforce_plan
procedure, and let the Database Engine find the optimal plan.
Tip
Alternatively, use the Queries With Forced Plans Query Store view to locate and unforce plans.
SQL Server provides all necessary views and procedures required to monitor performance and fix problems in Query Store.
In SQL Server 2016 (13.x), you can find plan choice regressions using Query Store system views. In SQL Server 2017 (14.x), the Database Engine detects and shows potential plan choice regressions and the recommended actions that should be applied in the sys.dm_db_tuning_recommendations (Transact-SQL)view. The view shows information about the problem, the importance of the issue, and details such as the identified query, the ID of the regressed plan, the ID of the plan that was used as baseline for comparison, and the Transact-SQL statement that can be executed to fix the problem.
type | description | datetime | score | details | ... |
---|---|---|---|---|---|
FORCE_LAST_GOOD_PLAN | CPU time changed from 4 ms to 14 ms | 3/17/2017 | 83 | queryId recommendedPlanId regressedPlanId T-SQL | |
FORCE_LAST_GOOD_PLAN | CPU time changed from 37 ms to 84 ms | 3/16/2017 | 26 | queryId recommendedPlanId regressedPlanId T-SQL |
Some columns from this view are described in the following list:
- Type of the recommended action -
FORCE_LAST_GOOD_PLAN
- Description that contains information why Database Engine thinks that this plan change is a potential performance regression
- Datetime when the potential regression is detected
- Score of this recommendation
- Details about the issues such as ID of the detected plan, ID of the regressed plan, ID of the plan that should be forced to fix the issue, Transact-SQLscript that might be applied to fix the issue, etc. Details are stored in JSON format
Use the following query to obtain a script that fixes the issue and additional information about the estimated gain:
Here is the result set.
reason | score | script | query_id | current plan_id | recommended plan_id | estimated_gain | error_prone |
---|---|---|---|---|---|---|---|
CPU time changed from 3 ms to 46 ms | 36 | EXEC sp_query_store_force_plan 12, 17; | 12 | 28 | 17 | 11.59 | 0 |
estimated_gain
represents the estimated number of seconds that would be saved if the recommended plan would be executed instead of the current plan. The recommended plan should be forced instead of the current plan if the gain is greater than 10 seconds. If there are more errors (for example, time-outs or aborted executions) in the current plan than in the recommended plan, the column error_prone
would be set to the value YES
. Error prone plan is another reason why the recommended plan should be forced instead of the current one.
Although Database Engine provides all information required to identify plan choice regressions; continuousmonitoring and fixing performance issues might be a tedious process. Automatic tuning makes this process much easier.
Note
Data in the sys.dm_db_tuning_recommendations DMV is not persisted between restarts of the SQL Server instance.
Automatic index management
In Azure SQL Database, index management is easy because Azure SQL Database learns about your workload and ensures that your data is always optimally indexed. Proper index design is crucial for optimal performance of your workload, and automatic index management can help you optimize your indexes. Automatic index management can either fix performance issues in incorrectly indexed databases, or maintain and improve indexes on the existing database schema. Automatic tuning in Azure SQL Database performs the following actions:
- Identifies indexes that could improve performance of your Transact-SQL queries that read data from the tables.
- Identifies the redundant indexes or indexes that were not used in longer period of time that could be removed. Removing unnecessary indexes improves performance of the queries that update data in tables.
Why do you need index management?
Indexes speed up some of your queries that read data from the tables; however, they can slow down the queries that update data. You need to carefully analyze when to create an index and what columns you need to include in the index. Some indexes might not be needed after some time. Therefore, you would need to periodically identify and drop the indexes that do not bring any benefits. If you ignore the unused indexes, performance of the queries that update data would be decreased without any benefit on the queries that read data. Unused indexes also affect overall performance of the system because additional updates require unnecessary logging.
Finding the optimal set of indexes that improve performance of the queries that read data from your tables and have minimal impact on updates might require continuous and complex analysis.
Azure SQL Database uses built-in intelligence and advanced rules that analyze your queries, identify indexes that would be optimal for your current workloads, and the indexes might be removed. Azure SQL Database ensures that you have a minimal necessary set of indexes that optimize the queries that read data, with the minimized impact on the other queries.
Automatic index management
In addition to detection, Azure SQL Database can automatically apply identified recommendations. If you find that the built-in rules improve the performance of your database, you might let Azure SQL Database automatically manage your indexes.
To enable automatic tuning in Azure SQL Database and allow the automatic tuning feature to fully manage your workload, see Enable automatic tuning in Azure SQL Database using Azure portal.
When the Azure SQL Database applies a CREATE INDEX or DROP INDEX recommendation, it automatically monitors the performance of the queries that are affected by the index. New index will be retained only if performances of the affected queries are improved. Dropped index will be automatically re-created if there are some queries that run slower due to the absence of the index.
Automatic index management considerations
Actions required to create necessary indexes in Azure SQL Database might consume resources and temporally affect workload performance. To minimize the impact of index creation on workload performance, Azure SQL Database finds the appropriate time window for any index management operation. Tuning action is postponed if the database needs resources to execute your workload, and started when the database has enough unused resources that can be used for the maintenance task. One important feature in automatic index management is a verification of the actions. When Azure SQL Database creates or drops index, a monitoring process analyzes performance of your workload to verify that the action improved the performance. If it didn't bring significant improvement - the action is immediately reverted. This way, Azure SQL Database ensures that automatic actions do not negatively impact performance of your workload. Indexes created by automatic tuning are transparent for the maintenance operation on the underlying schema. Schema changes such as dropping or renaming columns are not blocked by the presence of automatically created indexes. Indexes that are automatically created by Azure SQL Database are immediately dropped when related table or columns is dropped.
Alternative - manual index management
Without automatic index management, user would need to manually query sys.dm_db_missing_index_details (Transact-SQL) view or use the Performance Dashboard report in Management Studio to find indexes that might improve performance, create indexes using the details provided in this view, and manually monitor performance of the query. In order to find the indexes that should be dropped, users should monitor operational usage statistics of the indexes to find rarely used indexes.
Azure SQL Database simplifies this process. Azure SQL Database analyzes your workload, identifies the queries that could be executed faster with a new index, and identifies unused or duplicated indexes. Find more information about identification of indexes that should be changed at Find index recommendations in Azure portal.
See Also
ALTER DATABASE SET AUTOMATIC_TUNING (Transact-SQL)
sys.database_automatic_tuning_options (Transact-SQL)
sys.dm_db_tuning_recommendations (Transact-SQL)
sys.dm_db_missing_index_details (Transact-SQL)
sp_query_store_force_plan (Transact-SQL)
sp_query_store_unforce_plan (Transact-SQL)
sys.database_query_store_options (Transact-SQL)
JSON functions
Execution Plans
Monitor and Tune for Performance
Performance Monitoring and Tuning Tools
Monitoring Performance By Using the Query Store
Query Tuning Assistant
Azure SQL Database automatic tuning provides peak performance and stable workloads through continuous performance tuning based on AI and machine learning.
Automatic tuning is a fully managed intelligent performance service that uses built-in intelligence to continuously monitor queries executed on a database, and it automatically improves their performance. This is achieved through dynamically adapting database to the changing workloads and applying tuning recommendations. Automatic tuning learns horizontally from all databases on Azure through AI and it dynamically improves its tuning actions. The longer a database runs with automatic tuning on, the better it performs.
Azure SQL Database automatic tuning might be one of the most important features that you can enable to provide stable and peak performing database workloads.
What can automatic tuning do for you
- Automated performance tuning of Azure SQL databases
- Automated verification of performance gains
- Automated rollback and self-correction
- Tuning history
- Tuning action T-SQL scripts for manual deployments
- Proactive workload performance monitoring
- Scale out capability on hundreds of thousands of databases
- Positive impact to DevOps resources and the total cost of ownership
Safe, Reliable, and Proven
Tuning operations applied to databases in Azure SQL Database are fully safe for the performance of your most intense workloads. The system has been designed with care not to interfere with the user workloads. Automated tuning recommendations are applied only at the times of a low utilization. The system can also temporarily disable automatic tuning operations to protect the workload performance. In such case, 'Disabled by the system' message will be shown in Azure portal. Automatic tuning regards workloads with the highest resource priority.
Auto Tuning Index Azure Database Report Example Free
Automatic tuning mechanisms are mature and have been perfected on several million databases running on Azure. Automated tuning operations applied are verified automatically to ensure there is a positive improvement to the workload performance. Regressed performance recommendations are dynamically detected and promptly reverted. Through the tuning history recorded, there exists a clear trace of tuning improvements made to each Azure SQL Database.
Azure SQL Database automatic tuning is sharing its core logic with the SQL Server automatic tuning engine. For additional technical information on the built-in intelligence mechanism, see SQL Server automatic tuning.
For an overview of how automatic tuning works and for typical usage scenarios, see the embedded video:
Enable automatic tuning
You can enable automatic tuning for single and pooled databases in the Azure portal or using the ALTER DATABASE T-SQL statement. You enable automatic tuning for instance databases in a managed instance deployment using the ALTER DATABASE T-SQL statement.
Automatic tuning options
Automatic tuning options available in Azure SQL Database are:
Automatic tuning option | Single database and pooled database support | Instance database support |
---|---|---|
CREATE INDEX - Identifies indexes that may improve performance of your workload, creates indexes, and automatically verifies that performance of queries has improved. | Yes | No |
DROP INDEX - Identifies redundant and duplicate indexes daily, except for unique indexes, and indexes that were not used for a long time (>90 days). Please note that this option is not compatible with applications using partition switching and index hints. Dropping unused indexes is not supported for Premium and Business Critical service tiers. | Yes | No |
FORCE LAST GOOD PLAN (automatic plan correction) - Identifies SQL queries using execution plan that is slower than the previous good plan, and queries using the last known good plan instead of the regressed plan. | Yes | Yes |
Automatic tuning for single and pooled databases
Automatic tuning for single and pooled databases uses the CREATE INDEX, DROP INDEX, and FORCE LAST GOOD PLAN database advisor recommendations to optimize your database performance. For more information, see Database advisor recommendations in the Azure portal, in PowerShell, and in the REST API.
You can either manually apply tuning recommendations using the Azure portal or you can let automatic tuning autonomously apply tuning recommendations for you. The benefits of letting the system autonomously apply tuning recommendations for you is that it automatically validates there exists a positive gain to the workload performance, and if there is no significant performance improvement detected, it will automatically revert the tuning recommendation. Please note that in case of queries affected by tuning recommendations that are not executed frequently, the validation phase can take up to 72 hrs by design.
In case you are applying tuning recommendations through T-SQL, the automatic performance validation, and reversal mechanisms are not available. Recommendations applied in such way will remain active and shown in the list of tuning recommendations for 24-48 hrs. before the system automatically withdraws them. If you would like to remove a recommendation sooner, you can discard it from Azure portal.
Auto Tuning Index Azure Database Report Example Pdf
Automatic tuning options can be independently enabled or disabled per database, or they can be configured on SQL Database servers and applied on every database that inherits settings from the server. SQL Database servers can inherit Azure defaults for automatic tuning settings. Azure defaults at this time are set to FORCE_LAST_GOOD_PLAN is enabled, CREATE_INDEX is enabled, and DROP_INDEX is disabled.
Important
As of March, 2020 changes to Azure defaults for automatic tuning will take effect as follows:
- New Azure defaults will be FORCE_LAST_GOOD_PLAN = enabled, CREATE_INDEX = disabled, and DROP_INDEX = disabled.
- Existing servers with no automatic tuning preferences configured will be automatically configured to INHERIT the new Azure defaults. This applies to all customers currently having server settings for automatic tuning in an undefined state.
- New servers created will automatically be configured to INHERIT the new Azure defaults (unlike earlier when automatic tuning configuration was in an undefined state upon new server creation).
Configuring automatic tuning options on a server and inheriting settings for databases belonging to the parent server is a recommended method for configuring automatic tuning as it simplifies management of automatic tuning options for a large number of databases.
Auto Tuning Index Azure Database Report Example In Excel
To learn about building email notifications for automatic tuning recommendations, see Email notifications for automatic tuning.
Automatic tuning for instance databases
Automatic tuning for instance databases in a managed instance deployment only supports FORCE LAST GOOD PLAN. For more information about configuring automatic tuning options through T-SQL, see Automatic tuning introduces automatic plan correction and Automatic plan correction.
Next steps
- To learn about built-in intelligence used in automatic tuning, see Artificial Intelligence tunes Azure SQL databases.
- To learn how automatic tuning works under the hood, see Automatically indexing millions of databases in Microsoft Azure SQL Database.