Questions tagged [query-optimization]

For questions about making database queries run faster. PLEASE INCLUDE QUERY PLANS, AND TABLE AND INDEX DEFINITIONS in your question. Please use additional tags to specify your query language and server maker. Some of these questions may belong on https://DBA.stackexchange.com, especially if they involve reindexing, query plans, etc.

Please include table and index definitions for all relevant tables in your question. Please also include the query plan: the EXPLAIN output. Read on for more explanation.

If you use this is a good guide to pulling together the right information for your question.

SQL is a declarative language that can have many ways to find the same data. As a SQL programmer, you declare what you want from it, and it figures out how to get your result. Because of how databases are structured, and how they retrieve data, the same results can be retrieved in various ways that the query interpreter needs to choose between.

Using sargable predicates, using indexes, and optimizing how the query works can make it run more quickly. Restructuring the query, removing unnecessary portions of the query, or using different features of different Database Engines can all be helpful.

If the question is not (just) about how to write the query, but involves how to create new indexes, or how to optimize the server, you may get good answers on https://DBA.stackexchange.com.

Your database's query planner, also known as its query optimizer module attempts to determine the most efficient way to execute a given query by considering possible query plans.

You'll hear a lot about query planners in this tag. Each DBMS software product has its own query planner. Here's a description of one query planner.

How to ask a good query-optimization question

If you're asking "Why is my SQL query so slow?" or something like that, you need to do a little extra homework to get good answers from the Stack Overflow community. Please include this information.

What database and version?

Put this tag on your question along with the tag for your database: , . , or whatever database you use. Don't use more than one database tag, please. Optimization works somewhat differently from database to database. And, please mention the version of your database server.

How many rows and how fast?

Please figure out approximately how many rows you expect in your result set, and approximately how many rows are in each table involved with the query. Mention those numbers in your question. Please mention how long you hope your query needs to take, and how long it's taking. (That is, tell us what you mean by slow).

Don't oversimplify

Don't try to simplify your question by replacing your actual SELECT column lists with *. Using * in SELECT clauses can be harmful to query performance compared to specifying the list of columns you need.

Show your table definitions, with indexes

In your question include your table definitions and index definitions for each table and view involved in your query. Make sure your question shows all indexes in all relevant tables. When you're asking a performance question, please don't oversimplify your question by omitting indexes or columns you think might be irrelevant. The strangest things can be relevant to performance!

Very important: the names and data types of columns included in each index, with their order.

Show your database's explanation of how it satisfied your query

In your question please show the EXPLAIN PLAN output (it's just EXPLAIN or SHOWPLAN in some table server makes and models) for your query.

Things to know

  • Creating an appropriate index for a slow query is often, not always, a good solution to a problem. Answers on will often contain a suggestion for an index.

  • Avoid creating indexes on lots of single columns. Single-column indexes are often not very useful to speed up complex queries.

  • As your database grows you will, most likely, discover new slow queries. Operations can be trivial on hundreds of rows, but much more time-consuming on tens of thousands of rows. In a growing database, plan to review performance issues routinely.

  • If you use SSMS -- Microsoft's SQL Server Management Studio -- this tip is for you: Right-click in a query window, then select Show Actual Execution Plan, then run the query. The execution plan display sometimes recommends a new index to create.

Things you might do when optimizing queries

  1. Join ordering
  2. Query planning for nested SQL queries
  3. Cost estimation
  4. Parametric Query Optimization
  5. Multi-Objective Query Optimization
  6. Multi-Objective Parametric Query Optimization

Reference

Please be aware that query performance optimization is an inexact science. This is a good reference. http://use-the-index-luke.com/

7749 questions
35
votes
4 answers

What is the optimal way to compare dates in Microsoft SQL server?

I have a SQL datetime field in a very large table. It's indexed and needs to be queried. The problem is that SQL always stores the time component (even though it's always midnight), but the searches are to the day, rather than time. declare @dateVar…
Keith
  • 150,284
  • 78
  • 298
  • 434
33
votes
4 answers

MySQL "IN" queries terribly slow with subquery but fast with explicit values

I have a MySQL query (Ubu 10.04,Innodb, Core i7, 16Gb RAM, SSD drives, MySQL params optimized): SELECT COUNT(DISTINCT subscriberid) FROM em_link_data WHERE linkid in (SELECT l.id FROM em_link l WHERE l.campaignid = '2900' AND l.link != 'open') The…
Franco
  • 333
  • 1
  • 3
  • 5
32
votes
5 answers

SQL Server Index - Any improvement for LIKE queries?

We have a query that runs off a fairly large table that unfortunately needs to use LIKE '%ABC%' on a couple varchar fields so the user can search on partial names, etc. SQL Server 2005 Would adding an index on these varchar fields help any in terms…
schooner
  • 3,047
  • 8
  • 30
  • 39
32
votes
4 answers

How can I force a subquery to perform as well as a #temp table?

I am re-iterating the question asked by Mongus Pong Why would using a temp table be faster than a nested query? which doesn't have an answer that works for me. Most of us at some point find that when a nested query reaches a certain complexity it…
31
votes
2 answers

Should LIKE 'searchstr%' use an index?

I have a database with several fields : word_id — INTEGER PRIMARY_KEY word — TEXT ... And ~150k rows. Since this is a dictionary, I'm searching for a word with mask 'search_string%' using LIKE. It used to work, taking 15ms to find matching rows.…
nikans
  • 2,468
  • 1
  • 28
  • 35
31
votes
16 answers

SQL: Inner joining two massive tables

I have two massive tables with about 100 million records each and I'm afraid I needed to perform an Inner Join between the two. Now, both tables are very simple; here's the description: BioEntity table: BioEntityId (int) Name (nvarchar 4000,…
30
votes
13 answers

How to optimise this MySQL query? Millions of Rows

I have the following query: SELECT analytics.source AS referrer, COUNT(analytics.id) AS frequency, SUM(IF(transactions.status = 'COMPLETED', 1, 0)) AS sales FROM analytics LEFT JOIN transactions ON analytics.id =…
Abs
  • 56,052
  • 101
  • 275
  • 409
30
votes
2 answers

Is possible to reuse subqueries?

I'm having some problems trying to perform a query. I have two tables, one with elements information, and another one with records related with the elements of the first table. The idea is to get in the same row the element information plus several…
Gothmog
  • 871
  • 1
  • 8
  • 20
30
votes
5 answers

Which provides better performance one big join or multiple queries?

i have a table called orders. one column on order is customer_id i have a table called customers with 10 fields Given the two options if i want to build up an array of order objects and embedded in an order object is a customer object i have two…
leora
  • 188,729
  • 360
  • 878
  • 1,366
30
votes
2 answers

Optimizing MySQL LIKE '%string%' queries in innoDB

Having this table: CREATE TABLE `example` ( `id` int(11) unsigned NOT NULL auto_increment, `keywords` varchar(200) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; We would like to optimize the following query: SELECT id FROM example WHERE keywords…
jmserra
  • 1,296
  • 4
  • 18
  • 34
29
votes
3 answers

Finding blocking/locking queries in MS SQL (mssql)

Using sys.dm_os_wait_stats I have identified what I believe is a locking problem wait type waittime pct running ptc LCK_M_RS_S 2238.54 22.14 22.14 LCK_M_S 1980.59 19.59 41.73 Is there a way I can find the top…
29
votes
3 answers

How to reduce clustered index scan cost by using SQL query

How can I reduce the clustered index scan cost of below mentioned query DECLARE @PARAMVAL varchar(3) set @PARAMVAL = 'CTD' select * from MASTER_RECORD_TYPE where RECORD_TYPE_CODE=@PARAMVAL if I run the above query it was showing index scan 99 %…
user1494292
  • 399
  • 1
  • 5
  • 14
28
votes
8 answers

Can I optimize a SELECT DISTINCT x FROM hugeTable query by creating an index on column x?

I have a huge table, having a much smaller number (by orders of magnitude) of distinct values on some column x. I need to do a query like SELECT DISTINCT x FROM hugeTable, and I want to do this relatively fast. I did something like CREATE INDEX…
polygenelubricants
  • 376,812
  • 128
  • 561
  • 623
28
votes
3 answers

How can I analyse a Sqlite query execution?

I have a Sqlite database which I want to check the indexes are correct. MS SQL Analyser is great at breaking down the query execution and utilised indexes. Is there a similar tool for Sqlite?
Phil Hannent
  • 12,047
  • 17
  • 71
  • 118
27
votes
2 answers

How to properly index a many-many association table?

In a typical many-many arrangement like this... Movies Actors Movies_Actors ------ ------ ------------- movie_ID actor_ID FK_movie_ID title name FK_actor_ID ... how should the association table…
Russ
  • 10,835
  • 12
  • 42
  • 57