Strate SQL » Performance Tuning

Webcast Today – Using XML to Query Execution Plans

jstrate — Wed, 28 Jul 2010 14:00:00 +0000

I’ll be speaking for the SQL PASS Database Administration Virtual Chapter today at 12 PM Eastern time. The topic will be Using XML to Query Execution Plans. The abstract for the event is the following:

SQL Server stores its execution plans as XML in dynamic management views. The execution plans are a gold mine of information. From the whether or not the execution plan will rely on parallelism to what columns are requiring a key lookup after a non-clustered index seek. Through a the use of XML this information can be available at your fingertips to help determine the value and impact of an index and guide you in improving the performance of your SQL Server databases. In this session we’ll look at how you can begin to understand and query the structure of the execution plans in the procedure cache. Also, we’ll review how to uncover some potential performance issues that may be lurking in your SQL Server.

You can get to the event here.

Webcast Next Week – Using XML to Query Execution Plans

jstrate — Wed, 21 Jul 2010 14:00:00 +0000

I’ll be speaking for the SQL PASS Database Administration Virtual Chapter next week. It’ll be on July 28 at 12 PM Eastern time. The topic will be Using XML to Query Execution Plans. The abstract for the event is the following:

SQL Server stores its execution plans as XML in dynamic management views. The execution plans are a gold mine of information. From the whether or not the execution plan will rely on parallelism to what columns are requiring a key lookup after a non-clustered index seek. Through a the use of XML this information can be available at your fingertips to help determine the value and impact of an index and guide you in improving the performance of your SQL Server databases. In this session we’ll look at how you can begin to understand and query the structure of the execution plans in the procedure cache. Also, we’ll review how to uncover some potential performance issues that may be lurking in your SQL Server.

If you register for the event ahead of time you will be entered into a drawing brought to us from CA, our event sponsor. You can get to the event here.

PASSMN July Meeting Today

jstrate — Tue, 20 Jul 2010 14:00:00 +0000

Don’t forget the July PASSMN meeting is today. We have Itzik Ben-Gan (blog) coming in to talk about some query tuning tips.

Here are the meeting details…

Date and Time:

July 20, 2010, 5:00 PM CST

Location:

8300 Norman Center Drive, 9th Floor, Bloomington, MN 55437 (map)

Live Meeting

URL: https://www.livemeeting.com/cc/usergroups/join?id=7DGRT6&role=attend&pw=h*ZR%248%25%2Fd

Meeting ID: 7DGRT6

Entry Code: h*ZR$8%/d

Agenda:

5:00 –5:15 : Announcements
5:15 –5:25 : Sponsor
5:25 –6:45 : Query Tuning Tips (Itzik)
6:45 –7:00 : Survey Collection and Giveaways

Presentation:

Query Tuning Tips

Given a SQL Server querying problem there’s much that you can do to enable a good performing solution. Tuning involves arranging an optimal physical environment, e.g., by creating supporting indexes, as well as writing the query in a way that it would get an optimal execution plan. Many factors can affect the efficiency of the solution including the availability of indexes, data distribution and density, and others. In different scenarios, a different solution could be the most efficient for the same querying problem. Query tuning could be considered an art. This session will provide various tips to do efficient query tuning and demonstrate those through specific tuning examples.

Itzik Ben-Gan is a Mentor and Co-Founder of Solid Quality Mentors. A SQL Server Microsoft MVP (Most Valuable Professional) since 1999, Itzik has delivered numerous training events around the world focused on T-SQL Querying, Query Tuning and Programming. Itzik is the author of several books including Microsoft SQL Server 2008: T-SQL Fundamentals, Inside Microsoft SQL Server 2008: T-SQL Querying and Inside Microsoft SQL Server 2008: T-SQL Programming. He has written many articles for SQL Server Magazine as well as articles and whitepapers for MSDN. Itzik’s speaking activities include TechEd, DevWeek, SQLPASS, SQL Server Magazine Connections, various user groups around the world, and Solid Quality Mentors’ events to name a few. Itzik is the author of Solid Quality Mentors’ Advanced T-SQL Querying, Programming and Tuning and T-SQL Fundamentals courses along with being a primary resource within the company for their T-SQL related activities

PASSMN July Meeting Next Week

jstrate — Tue, 13 Jul 2010 13:00:00 +0000

The July PASSMN meeting is is coming up next week. If you’ve not registered now’s a great time to get your RSVP in. We have Itzik Ben-Gan (blog) coming in to talk about some query tuning tips.

Here are the meeting details…

Date and Time:

July 20, 2010, 5:00 PM CST

Location:

8300 Norman Center Drive, 9th Floor, Bloomington, MN 55437 (map)

Live Meeting

URL: https://www.livemeeting.com/cc/usergroups/join?id=7DGRT6&role=attend&pw=h*ZR%248%25%2Fd

Meeting ID: 7DGRT6

Entry Code: h*ZR$8%/d

Agenda:

5:00 –5:15 : Announcements
5:15 –5:25 : Sponsor
5:25 –6:45 : Query Tuning Tips (Itzik)
6:45 –7:00 : Survey Collection and Giveaways

Presentation:

Query Tuning Tips

Given a SQL Server querying problem there’s much that you can do to enable a good performing solution. Tuning involves arranging an optimal physical environment, e.g., by creating supporting indexes, as well as writing the query in a way that it would get an optimal execution plan. Many factors can affect the efficiency of the solution including the availability of indexes, data distribution and density, and others. In different scenarios, a different solution could be the most efficient for the same querying problem. Query tuning could be considered an art. This session will provide various tips to do efficient query tuning and demonstrate those through specific tuning examples.

Itzik Ben-Gan is a Mentor and Co-Founder of Solid Quality Mentors. A SQL Server Microsoft MVP (Most Valuable Professional) since 1999, Itzik has delivered numerous training events around the world focused on T-SQL Querying, Query Tuning and Programming. Itzik is the author of several books including Microsoft SQL Server 2008: T-SQL Fundamentals, Inside Microsoft SQL Server 2008: T-SQL Querying and Inside Microsoft SQL Server 2008: T-SQL Programming. He has written many articles for SQL Server Magazine as well as articles and whitepapers for MSDN. Itzik’s speaking activities include TechEd, DevWeek, SQLPASS, SQL Server Magazine Connections, various user groups around the world, and Solid Quality Mentors’ events to name a few. Itzik is the author of Solid Quality Mentors’ Advanced T-SQL Querying, Programming and Tuning and T-SQL Fundamentals courses along with being a primary resource within the company for their T-SQL related activities

Please RSVP here if you will be attending in person, this helps us understand how many people to tell the sponsor to get food for.

Webcast Today – Performance Impacts Related to Different Function Types

jstrate — Tue, 06 Jul 2010 14:00:00 +0000

I’ll be speaking for the SQL PASS Performance Virtual Chapter later today starting at 12 PM Eastern time. The topic will be Performance Impacts Related to Different Function Types. The abstract for the event is the following:

User defined functions provide a means to encapsulate business logic in the database tier. Often the purpose of the encapsulation is to provide standard method access segments of data within the database. Unfortunately, not all methods of creating user defined functions are equal. In this session we’ll review the types of user defined functions and investigate the performance impact in selecting the different types

Goals:

Identify purposes for creating user defined functions

Discuss the types of user-defined functions

Demonstrate performance impact in selecting different types of functions

You can get to the event here.

July PASSMN Meeting Announced

jstrate — Tue, 29 Jun 2010 17:00:00 +0000

Sometimes when we are putting together the user group meetings things change. Opportunities arise and we get a chance to take a cool meeting and turn it into a great meeting. That happened with this months meeting and it’s going to be great.

A couple things are different for this month… time and topic. This month instead of meeting at 3:00 PM we are meeting at 5:00 PM. And the reason for the change is that we have Itzik Ben-Gan (blog) stopping by for a presentation.

The Presentation

This month he’ll be teaching a course at Benchmark Learning. His course is Advanced T-SQL Querying, Programming and Tuning for SQL Server 2005 & 2008. If you want more than what you suspect will be at the meeting, then this class is for you.

What can you suspect will be at the meeting this month? Well, how about this?

Query Tuning Tips

Given a SQL Server querying problem there’s much that you can do to enable a good performing solution. Tuning involves arranging an optimal physical environment, e.g., by creating supporting indexes, as well as writing the query in a way that it would get an optimal execution plan. Many factors can affect the efficiency of the solution including the availability of indexes, data distribution and density, and others. In different scenarios, a different solution could be the most efficient for the same querying problem. Query tuning could be considered an art. This session will provide various tips to do efficient query tuning and demonstrate those through specific tuning examples.

Pretty cool stuff, eh?

And the Rest of the Details…

This month’s meeting sponsor is Digineer. They’ll be providing the food and beverages. Digineer happens to be my employer and if you’re local and looking for a company with talent and skill this is a good place to look.

Date: July 20, 2010
Time: 5:00 PM – 7:00 PM
Location: 8300 Norman Center Drive, 9th Floor, Bloomington, MN 55437
Live Meeting

URL: https://www.livemeeting.com/cc/usergroups/join?id=7DGRT6&role=attend&pw=h*ZR%248%25%2Fd

Meeting ID: 7DGRT6

Entry Code: h*ZR$8%/d

The all important RSVP link – please do RSVP so that we’ll have enough food for the event.

Webcast Next Week – Performance Impacts Related to Different Function Types

jstrate — Mon, 28 Jun 2010 17:00:00 +0000

I’ll be speaking for the SQL PASS Performance Virtual Chapter next week. It’ll be on July 6 at 12 PM Eastern time. The topic will be Performance Impacts Related to Different Function Types. The abstract for the event is the following:

User defined functions provide a means to encapsulate business logic in the database tier. Often the purpose of the encapsulation is to provide standard method access segments of data within the database. Unfortunately, not all methods of creating user defined functions are equal. In this session we’ll review the types of user defined functions and investigate the performance impact in selecting the different types

Goals:

Identify purposes for creating user defined functions

Discuss the types of user-defined functions

Demonstrate performance impact in selecting different types of functions

You can get to the event here.

Index Those Foreign Keys

jstrate — Fri, 18 Jun 2010 21:00:59 +0000

Today started with some quality time getting to know a deadlock that had occurred. While working through the deadlock, I noticed that there were a number of foreign key relationships that weren’t indexed on the parent side of the relationship.

I am going to skip over the why to index foreign keys and save that for a later point when I have more time to go through it with some really pretty pictures. Today though, I want to share the scripts that I put together to look for these situations and help prevent issues related to them.

Brute Force Indexing

This first script is a brute force attack on this need. If you set the output for text results in SQL Server Management Studio (SSMS) you’ll get a script with all of the indexes you’ll need to cover all of your foreign key relationships.


SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
SET NOCOUNT ON

;WITH cIndexes
AS (
SELECT i.object_id
,i.name
,(SELECT QUOTENAME(ic.column_id,'(')
FROM sys.index_columns ic
WHERE i.object_id = ic.object_id
AND i.index_id = ic.index_id
AND is_included_column = 0
ORDER BY key_ordinal ASC
FOR XML PATH('')) AS indexed_compare
FROM sys.indexes i
), cForeignKeys
AS (
SELECT fk.name AS foreign_key_name
,'PARENT' as foreign_key_type
,fkc.parent_object_id AS object_id
,STUFF((SELECT ', ' + QUOTENAME(c.name)
FROM sys.foreign_key_columns ifkc
INNER JOIN sys.columns c ON ifkc.parent_object_id = c.object_id AND ifkc.parent_column_id = c.column_id
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')), 1, 2, '') AS fk_columns
,(SELECT QUOTENAME(ifkc.parent_column_id,'(')
FROM sys.foreign_key_columns ifkc
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')) AS fk_columns_compare
FROM sys.foreign_keys fk
INNER JOIN sys.foreign_key_columns fkc ON fk.object_id = fkc.constraint_object_id
WHERE fkc.constraint_column_id = 1
UNION ALL
SELECT fk.name AS foreign_key_name
,'REFERENCED' as foreign_key_type
,fkc.referenced_object_id AS object_id
,STUFF((SELECT ', ' + QUOTENAME(c.name)
FROM sys.foreign_key_columns ifkc
INNER JOIN sys.columns c ON ifkc.referenced_object_id = c.object_id AND ifkc.referenced_column_id = c.column_id
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')), 1, 2, '') AS fk_columns
,(SELECT QUOTENAME(ifkc.referenced_column_id,'(')
FROM sys.foreign_key_columns ifkc
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')) AS fk_columns_compare
FROM sys.foreign_keys fk
INNER JOIN sys.foreign_key_columns fkc ON fk.object_id = fkc.constraint_object_id
WHERE fkc.constraint_column_id = 1
), cRowCount
AS (
SELECT object_id
,SUM(row_count) AS row_count
FROM sys.dm_db_partition_stats ps
WHERE index_id IN (1,0)
GROUP BY object_id
)
SELECT
'--Missing foreign key index for '+fk.foreign_key_name+CHAR(13)+CHAR(10)+'GO'+CHAR(13)+CHAR(10)+
+'CREATE NONCLUSTERED INDEX FKIX_'+OBJECT_NAME(fk.object_id)+'_'+REPLACE(REPLACE(REPLACE(REPLACE(fk.fk_columns,',',''),'[',''),']',''),' ','')
+CHAR(13)+CHAR(10)+
+'ON [dbo].['+OBJECT_NAME(fk.object_id)+'] ('+fk.fk_columns+')'+CHAR(13)+CHAR(10)+
+'GO'+CHAR(13)+CHAR(10)+CHAR(13)+CHAR(10)
FROM cForeignKeys fk
INNER JOIN cRowCount rc ON fk.object_id = rc.object_id
LEFT OUTER JOIN cIndexes i ON fk.object_id = i.object_id AND i.indexed_compare LIKE fk.fk_columns_compare + '%'
WHERE i.name IS NULL
ORDER BY OBJECT_NAME(fk.object_id), fk.fk_columns

Foreign Key Monitoring

This second script accommodates for those situations when you may not want to just index every foreign key that is out there. Maybe there’s a really old table in the database with a foreign key relationship that just doesn’t matter any more. Is it worth indexing along a vector that won’t lead to any performance impact – either negative or positive? Most likely not.

For this script, the results output a list of foreign keys relationships that are not fully indexed. Included in the result script is a column with XML data in it that contains a script for creating an index. You may notice that the format for this is very similar to the schema created when outputtingmmissing Indexes from execution plans.


SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED

;WITH cIndexes
AS (
SELECT i.object_id
,i.name
,(SELECT QUOTENAME(ic.column_id,'(')
FROM sys.index_columns ic
WHERE i.object_id = ic.object_id
AND i.index_id = ic.index_id
AND is_included_column = 0
ORDER BY key_ordinal ASC
FOR XML PATH('')) AS indexed_compare
FROM sys.indexes i
), cForeignKeys
AS (
SELECT fk.name AS foreign_key_name
,'PARENT' as foreign_key_type
,fkc.parent_object_id AS object_id
,STUFF((SELECT ', ' + QUOTENAME(c.name)
FROM sys.foreign_key_columns ifkc
INNER JOIN sys.columns c ON ifkc.parent_object_id = c.object_id AND ifkc.parent_column_id = c.column_id
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')), 1, 2, '') AS fk_columns
,(SELECT QUOTENAME(ifkc.parent_column_id,'(')
FROM sys.foreign_key_columns ifkc
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')) AS fk_columns_compare
FROM sys.foreign_keys fk
INNER JOIN sys.foreign_key_columns fkc ON fk.object_id = fkc.constraint_object_id
WHERE fkc.constraint_column_id = 1
UNION ALL
SELECT fk.name AS foreign_key_name
,'REFERENCED' as foreign_key_type
,fkc.referenced_object_id AS object_id
,STUFF((SELECT ', ' + QUOTENAME(c.name)
FROM sys.foreign_key_columns ifkc
INNER JOIN sys.columns c ON ifkc.referenced_object_id = c.object_id AND ifkc.referenced_column_id = c.column_id
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')), 1, 2, '') AS fk_columns
,(SELECT QUOTENAME(ifkc.referenced_column_id,'(')
FROM sys.foreign_key_columns ifkc
WHERE fk.object_id = ifkc.constraint_object_id
ORDER BY ifkc.constraint_column_id
FOR XML PATH('')) AS fk_columns_compare
FROM sys.foreign_keys fk
INNER JOIN sys.foreign_key_columns fkc ON fk.object_id = fkc.constraint_object_id
WHERE fkc.constraint_column_id = 1
), cRowCount
AS (
SELECT object_id
,SUM(row_count) AS row_count
FROM sys.dm_db_partition_stats ps
WHERE index_id IN (1,0)
GROUP BY object_id
)
SELECT
fk.foreign_key_name
,OBJECT_NAME(fk.object_id) AS fk_table_name
,fk.fk_columns
,rc.row_count AS row_count
,CAST(' +CHAR(13)+CHAR(10)+'GO'+CHAR(13)+CHAR(10)+
+'CREATE NONCLUSTERED INDEX []'+CHAR(13)+CHAR(10)+
+'ON [dbo].['+OBJECT_NAME(fk.object_id)+'] ('+fk.fk_columns+')'+CHAR(13)+CHAR(10)+
+'GO'+CHAR(13)+CHAR(10)+'--?>' AS xml) foreign_key_index_schema
FROM cForeignKeys fk
INNER JOIN cRowCount rc ON fk.object_id = rc.object_id
LEFT OUTER JOIN cIndexes i ON fk.object_id = i.object_id AND i.indexed_compare LIKE fk.fk_columns_compare + '%'
WHERE i.name IS NULL
ORDER BY OBJECT_NAME(fk.object_id), fk.fk_columns

Closing Up

The DDL schema output in these scripts is very basic. It doesn’t account for potentially important things like partitions and filegroups. Obviously, you’ll need to modify this for your own environment and don’t just run this on production.

I see a lot of potential in these scripts and am planning to include them as part of preparing for releases when I am clients. A good way to dot the i’s and cross the t’s.

Individual results may vary. No Legos were harmed in the writing on this post.

Searching for Plans

jstrate — Mon, 03 May 2010 22:00:00 +0000

A while back I wrote a post on a procedure I created for the DBADiagnostics database that I talk about from time to time. This procedure allowed users to search the procedure cache to find plans for procedures by database name and object name. This had worked out pretty good until I noticed something the other day.

The procedure is using the sys.dm_exec_query_stats as the based table for the query to build itself upon. The trouble with using this DMV is that it doesn’t have all of the plans listed from the procedure cache. Only those that currently have stats stored for sql_handles.

About the same time I started writing this post, I noticed a post by Adam Machanic (twitter | blog) in which he puts out a warning to those using sys.dm_exec_query_stats. In that post, he discusses how the ALTER TABLE statement can result in a batch missing this DMV.

Considering all this lends itself to the question, is there a better DMV that can be used to find plans in the procedure cache?

Use sys.dm_exec_cached_plans Instead

Another DMV that has this information in it is the DMV sys.dm_exec_cached_plans. This DMV is designed to provide a list of all cached plans in the procedure cache. To illustrate the difference, lets take a look at the following two queries.


SELECT COUNT(DISTINCT plan_handle)
FROM sys.dm_exec_cached_plans
GO 

SELECT COUNT(DISTINCT plan_handle)
FROM sys.dm_exec_query_stats
GO

On my system, the following results are returned:

The first value is the number of plan_handles in sys.dm_exec_cached_plans. The second is the number of plan_handles in sys.dm_exec_query_stats. There is a substantial difference between the two. This is something you’d be expecting based on the information at the start of this post.

Query to Find Cached Plans

Now that I’ve outlined the issues with my original post, lets take a look at a script that will query the procedure cache to look for plans stored for stored procedures and other database objects.


SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
GO

DECLARE @DatabaseName sysname
    ,@ObjectName sysname 

SELECT @DatabaseName = 'msdb'
    ,@ObjectName = 'sp_jobhistory_row_limiter';

WITH PlanSearch
AS (
      SELECT qp.dbid
            ,qp.objectid
            ,DB_NAME(qp.dbid) as DatabaseName
            ,OBJECT_NAME(qp.objectid, qp.dbid) as ObjectName
            ,cp.usecounts
            ,cp.plan_handle
      FROM sys.dm_exec_cached_plans cp
            CROSS APPLY sys.dm_exec_text_query_plan(cp.plan_handle, DEFAULT, DEFAULT) qp
    WHERE cp.cacheobjtype = 'Compiled Plan'
      AND (DB_NAME(qp.dbid) = @DatabaseName OR NULLIF(@DatabaseName,'') IS NULL)
    AND (OBJECT_NAME(qp.objectid, qp.dbid) = @ObjectName OR NULLIF(@ObjectName, '') IS NULL)
)
,PlansAndStats
AS (
      SELECT ps.DatabaseName
        ,ps.ObjectName
        ,ps.usecounts -- Use in place of qs.execution_count for whole plan count
        ,CAST(SUM(qs.total_worker_time)/(ps.usecounts*1.) as decimal(12,2)) AS avg_cpu_time
        ,CAST(SUM(qs.total_logical_reads + qs.total_logical_writes)/(ps.usecounts*1.) as decimal(12,2)) AS avg_io
        ,SUM(qs.total_elapsed_time)/(ps.usecounts)/1000 as avg_elapsed_time_ms
        ,ps.plan_handle
    FROM PlanSearch ps
            LEFT OUTER JOIN sys.dm_exec_query_stats qs ON ps.plan_handle = qs.plan_handle
    GROUP BY ps.DatabaseName
        ,ps.ObjectName
        ,ps.usecounts
            ,ps.plan_handle
)
SELECT ps.DatabaseName
    ,ps.ObjectName
      ,ps.usecounts
      ,ps.avg_cpu_time
      ,ps.avg_io
      ,ps.avg_elapsed_time_ms
      ,qp.query_plan
      ,ps.plan_handle
FROM PlansAndStats ps
      CROSS APPLY sys.dm_exec_query_plan(ps.plan_handle) qp

Executing the script above provides the following output:

The output includes the name of the object being sought and a link to the plan from the procedure cache. Included with this is some basic performance information from sys.dm_exec_query_stats that can be useful in determining performance variations between the plans returned.

Procedure to Find Plans

For those that like to take these scripts and build stored procedures for them. I’ve also included a script below that includes this information.

Utility.CachedPlanSearch.txt

Changing A Performance Counter Log Format

jstrate — Sun, 02 May 2010 17:37:00 +0000

Ever come across performance counter log that isn’t in your desired format? This is a situation I’ve been in a number of times. And wouldn’t you know, I usually space off the command line tool that I can use to do this.

The gem of a tool that you can use to change performance log formats is called relog. There are a number parameters that can be passed into the tool which are detailed on the TechNet page for relog.

In a simple example, supposed you have a performance counter log named SQLStats.blg that was in a binary format. And suppose ever further that you needed this log to be in a CSV format.

Executing relog with the following parameters would complete this task above.

relog "SQLStats.blg" -f CSV -o "SQLStats.csv"