Questions tagged [query-optimization]

For questions about making database queries run faster. PLEASE INCLUDE QUERY PLANS, AND TABLE AND INDEX DEFINITIONS in your question. Please use additional tags to specify your query language and server maker. Some of these questions may belong on https://DBA.stackexchange.com, especially if they involve reindexing, query plans, etc.

Please include table and index definitions for all relevant tables in your question. Please also include the query plan: the EXPLAIN output. Read on for more explanation.

If you use this is a good guide to pulling together the right information for your question.

SQL is a declarative language that can have many ways to find the same data. As a SQL programmer, you declare what you want from it, and it figures out how to get your result. Because of how databases are structured, and how they retrieve data, the same results can be retrieved in various ways that the query interpreter needs to choose between.

Using sargable predicates, using indexes, and optimizing how the query works can make it run more quickly. Restructuring the query, removing unnecessary portions of the query, or using different features of different Database Engines can all be helpful.

If the question is not (just) about how to write the query, but involves how to create new indexes, or how to optimize the server, you may get good answers on https://DBA.stackexchange.com.

Your database's query planner, also known as its query optimizer module attempts to determine the most efficient way to execute a given query by considering possible query plans.

You'll hear a lot about query planners in this tag. Each DBMS software product has its own query planner. Here's a description of one query planner.

How to ask a good query-optimization question

If you're asking "Why is my SQL query so slow?" or something like that, you need to do a little extra homework to get good answers from the Stack Overflow community. Please include this information.

What database and version?

Put this tag on your question along with the tag for your database: , . , or whatever database you use. Don't use more than one database tag, please. Optimization works somewhat differently from database to database. And, please mention the version of your database server.

How many rows and how fast?

Please figure out approximately how many rows you expect in your result set, and approximately how many rows are in each table involved with the query. Mention those numbers in your question. Please mention how long you hope your query needs to take, and how long it's taking. (That is, tell us what you mean by slow).

Don't oversimplify

Don't try to simplify your question by replacing your actual SELECT column lists with *. Using * in SELECT clauses can be harmful to query performance compared to specifying the list of columns you need.

Show your table definitions, with indexes

In your question include your table definitions and index definitions for each table and view involved in your query. Make sure your question shows all indexes in all relevant tables. When you're asking a performance question, please don't oversimplify your question by omitting indexes or columns you think might be irrelevant. The strangest things can be relevant to performance!

Very important: the names and data types of columns included in each index, with their order.

Show your database's explanation of how it satisfied your query

In your question please show the EXPLAIN PLAN output (it's just EXPLAIN or SHOWPLAN in some table server makes and models) for your query.

Things to know

  • Creating an appropriate index for a slow query is often, not always, a good solution to a problem. Answers on will often contain a suggestion for an index.

  • Avoid creating indexes on lots of single columns. Single-column indexes are often not very useful to speed up complex queries.

  • As your database grows you will, most likely, discover new slow queries. Operations can be trivial on hundreds of rows, but much more time-consuming on tens of thousands of rows. In a growing database, plan to review performance issues routinely.

  • If you use SSMS -- Microsoft's SQL Server Management Studio -- this tip is for you: Right-click in a query window, then select Show Actual Execution Plan, then run the query. The execution plan display sometimes recommends a new index to create.

Things you might do when optimizing queries

  1. Join ordering
  2. Query planning for nested SQL queries
  3. Cost estimation
  4. Parametric Query Optimization
  5. Multi-Objective Query Optimization
  6. Multi-Objective Parametric Query Optimization

Reference

Please be aware that query performance optimization is an inexact science. This is a good reference. http://use-the-index-luke.com/

7749 questions
12
votes
2 answers

T-SQL query performance puzzle: Why does using a variable make a difference?

I'm trying to optimize a complex SQL query and getting wildly different results when I make seemingly inconsequential changes. For example, this takes 336 ms to run: Declare @InstanceID int set @InstanceID=1; With myResults as ( Select …
Herb Caudill
  • 50,043
  • 39
  • 124
  • 173
12
votes
10 answers

Sum for multiple date ranges in a single query?

I have the following query: SELECT SUM("balance_transactions"."fee") AS sum_id FROM "balance_transactions" JOIN charges ON balance_transactions.source = charges.balance_id WHERE "balance_transactions"."account_id" = 6 AND…
Shpigford
  • 24,748
  • 58
  • 163
  • 252
12
votes
2 answers

Change varchar to boolean in PostgreSQL

I've started working on a project where there is a fairly large table (about 82,000,000 rows) that I think is very bloated. One of the fields is defined as: consistency character varying NOT NULL DEFAULT 'Y'::character varying It's used as a…
David S
  • 12,967
  • 12
  • 55
  • 93
11
votes
2 answers

Why query optimizer selects completely different query plans?

Let us have the following table in SQL Server 2016 -- generating 1M test table with four attributes WITH x AS ( SELECT n FROM (VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) v(n) ), t1 AS ( SELECT ones.n + 10 * tens.n + 100 * hundreds.n + 1000…
Radim Bača
  • 10,646
  • 1
  • 19
  • 33
11
votes
3 answers

Disable spark catalyst optimizer

To give some background, I am trying to run TPCDS benchmark on Spark with and without Spark's catalyst optimizer. For complicated queries on smaller datasets, we might be spending more time optimizing the plans than actually executing the plans.…
11
votes
6 answers

How do you fix queries that only run slow until they're cached

I have some queries that are causing timeouts in our live environment. (>30 seconds) If I run profiler and grab the exact SQL being run and run it from Management Studio then they take a long time to run the first time and then drop to a few hundred…
Robin Day
  • 100,552
  • 23
  • 116
  • 167
11
votes
8 answers

Mysql improve SELECT speed

I'm currently trying to improve the speed of SELECTS for a MySQL table and would appreciate any suggestions on ways to improve it. We have over 300 million records in the table and the table has the structure tag, date, value. The primary key is a…
11
votes
3 answers

Does the order of columns in a query matter?

When selecting columns from a MySQL table, is performance affected by the order that you select the columns as compared to their order in the table (not considering indexes that may cover the columns)? For example, you have a table with rows uid,…
James Simpson
  • 13,488
  • 26
  • 83
  • 108
11
votes
6 answers

are there any Query Optimization Tools for SQL Server?

Are there any tools that do Query Optimizations for SQL Server 2005 and above? I've searched & didn't find anything. What do you recommend? If this Question was repeated before you can close it but I didn't see anything similar
Saif al Harthi
  • 2,948
  • 1
  • 21
  • 26
11
votes
6 answers

How to optimize query with many joins?

I have simple but long query which count the content of the result it takes about 14 seconds. the count itself on the main table takes less than a second but after multiple join the delay is too high as follow Select Count(Distinct visits.id) As…
Eslam Sameh Ahmed
  • 3,792
  • 2
  • 24
  • 40
11
votes
5 answers

2gb table with 10 million rows, late pagination select is slow

I have table in MySQL with 10 million rows with 2 GB data selecting IN LIFO format data is slow Table engine is = InnoDB table has one primary key and one unique key SELECT * FROM link LIMIT 999999 , 50; how I improve the performance of the…
Saurabh Chandra Patel
  • 12,712
  • 6
  • 88
  • 78
11
votes
2 answers

Query with ORDER BY is 13 times as slow when I add LIMIT 1

I have this query (in postgresql): SELECT "table_1".* FROM "table_1" INNER JOIN "join_table" ON "table_1"."id" = "join_table"."table_1_id" WHERE "join_table"."table_2_id" = 650727 ORDER BY table_1.created_at DESC LIMIT 1 Which returns 1 result,…
nzifnab
  • 15,876
  • 3
  • 50
  • 65
11
votes
2 answers

Hibernate @OneToOne executes multiple queries even with @Fetch(FetchMode.JOIN)

Consider and Employee and Address relationship. There is a One-to-one mapping between Employee and Address. Following are models: @Entity @Table(name = "Address") public class Address { @Id @GeneratedValue @Column(name = "addressId") …
Sanket Meghani
  • 885
  • 2
  • 14
  • 22
11
votes
1 answer

What is an automatic covering index?

When using EXPLAIN QUERY PLAN in SQLite 3 it sometimes gives me output such as SEARCH TABLE staff AS s USING AUTOMATIC COVERING INDEX (is_freelancer=? AND sap=?) (~6 rows) Where does the index come from and what does it do? The table has no…
Tamlyn
  • 22,122
  • 12
  • 111
  • 127
11
votes
2 answers

Oracle not using index when joining

I am very new to index and explain plans, so please bear with me! I am trying to tune a query but I am having issues. I have two tables: SKU ------ SKUIDX (Unique index) CLRIDX (Index) .. .. IMPCOST_CLR ----------- ICCIDX (Unique index) CLRIDX…
Lock
  • 5,422
  • 14
  • 66
  • 113