Questions tagged [query-optimization]

For questions about making database queries run faster. PLEASE INCLUDE QUERY PLANS, AND TABLE AND INDEX DEFINITIONS in your question. Please use additional tags to specify your query language and server maker. Some of these questions may belong on https://DBA.stackexchange.com, especially if they involve reindexing, query plans, etc.

Please include table and index definitions for all relevant tables in your question. Please also include the query plan: the EXPLAIN output. Read on for more explanation.

If you use this is a good guide to pulling together the right information for your question.

SQL is a declarative language that can have many ways to find the same data. As a SQL programmer, you declare what you want from it, and it figures out how to get your result. Because of how databases are structured, and how they retrieve data, the same results can be retrieved in various ways that the query interpreter needs to choose between.

Using sargable predicates, using indexes, and optimizing how the query works can make it run more quickly. Restructuring the query, removing unnecessary portions of the query, or using different features of different Database Engines can all be helpful.

If the question is not (just) about how to write the query, but involves how to create new indexes, or how to optimize the server, you may get good answers on https://DBA.stackexchange.com.

Your database's query planner, also known as its query optimizer module attempts to determine the most efficient way to execute a given query by considering possible query plans.

You'll hear a lot about query planners in this tag. Each DBMS software product has its own query planner. Here's a description of one query planner.

How to ask a good query-optimization question

If you're asking "Why is my SQL query so slow?" or something like that, you need to do a little extra homework to get good answers from the Stack Overflow community. Please include this information.

What database and version?

Put this tag on your question along with the tag for your database: , . , or whatever database you use. Don't use more than one database tag, please. Optimization works somewhat differently from database to database. And, please mention the version of your database server.

How many rows and how fast?

Please figure out approximately how many rows you expect in your result set, and approximately how many rows are in each table involved with the query. Mention those numbers in your question. Please mention how long you hope your query needs to take, and how long it's taking. (That is, tell us what you mean by slow).

Don't oversimplify

Don't try to simplify your question by replacing your actual SELECT column lists with *. Using * in SELECT clauses can be harmful to query performance compared to specifying the list of columns you need.

Show your table definitions, with indexes

In your question include your table definitions and index definitions for each table and view involved in your query. Make sure your question shows all indexes in all relevant tables. When you're asking a performance question, please don't oversimplify your question by omitting indexes or columns you think might be irrelevant. The strangest things can be relevant to performance!

Very important: the names and data types of columns included in each index, with their order.

Show your database's explanation of how it satisfied your query

In your question please show the EXPLAIN PLAN output (it's just EXPLAIN or SHOWPLAN in some table server makes and models) for your query.

Things to know

  • Creating an appropriate index for a slow query is often, not always, a good solution to a problem. Answers on will often contain a suggestion for an index.

  • Avoid creating indexes on lots of single columns. Single-column indexes are often not very useful to speed up complex queries.

  • As your database grows you will, most likely, discover new slow queries. Operations can be trivial on hundreds of rows, but much more time-consuming on tens of thousands of rows. In a growing database, plan to review performance issues routinely.

  • If you use SSMS -- Microsoft's SQL Server Management Studio -- this tip is for you: Right-click in a query window, then select Show Actual Execution Plan, then run the query. The execution plan display sometimes recommends a new index to create.

Things you might do when optimizing queries

  1. Join ordering
  2. Query planning for nested SQL queries
  3. Cost estimation
  4. Parametric Query Optimization
  5. Multi-Objective Query Optimization
  6. Multi-Objective Parametric Query Optimization

Reference

Please be aware that query performance optimization is an inexact science. This is a good reference. http://use-the-index-luke.com/

7749 questions
11
votes
1 answer

MySQL: comparison of integer value and string field with index

Table a_table has index on string_column. I have a query: SELECT * FROM a_table WHERE string_column = 10; I used EXPLAIN to find that no indexes are used. Why? Could you help me with MySQL documentation link? Updated: Sandbox (SQL Fiddle)
Dmitry
  • 7,457
  • 12
  • 57
  • 83
11
votes
5 answers

Is storing counts of database record redundant?

I'm using Rails and MySQL, and have an efficiency question based on row counting. I have a Project model that has_many :donations. I want to count the number of unique donors for a project. Is having a field in the projects table called num_donors,…
nfm
  • 19,689
  • 15
  • 60
  • 90
11
votes
2 answers

Role of selectivity in index scan/seek

I have been reading in many SQL books and articles that selectivity is an important factor in creating index. If a column has low selectivity, an index seek does more harm that good. But none of the articles explain why. Can anybody explain why it…
SexyBeast
  • 7,913
  • 28
  • 108
  • 196
11
votes
4 answers

Oracle <> , != , ^= operators

I want to know the difference of those operators, mainly their performance difference. I have had a look at Difference between <> and != in SQL, it has no performance related information. Then I found this on dba-oracle.com, it suggests that in 10.2…
Junchen Liu
  • 5,435
  • 10
  • 51
  • 62
10
votes
2 answers

SQL magic - query shouldn't take 15 hours, but it does

Ok, so i have one really monstrous MySQL table (900k records, 180 MB total), and i want to extract from subgroups records with higher date_updated and calculate weighted average in each group. The calculation runs for ~15 hours, and i have a strong…
Kuroki Kaze
  • 8,161
  • 4
  • 36
  • 48
10
votes
2 answers

How can I select the nearest value less-than and greater-than a given value efficiently?

I have two tables, one for values one for location and am trying to interpolate location. The tables have been simplified to the following: CREATE TABLE value( Timestamp DATETIME2, Value float NOT NULL, PRIMARY KEY(Timestamp) ); CREATE…
10
votes
1 answer

select top 10 ... and select top 30 follows different execution plan

During query optimization I encounted a strange behaviour of sql server (Sql Server 2008 R2 Enterprise). I created several indexes on tables, as well as some indexed views. I have two queries, for example: select top 10…
objectbox
  • 1,281
  • 1
  • 11
  • 13
10
votes
3 answers

MySQL - Views - Super slow query

This is a weird one. I am trying to use Views in MySQL (I'm reasonably new to MySQL with more experience with Sybase and SQL Server). Any way this new project we are using MySQL as it seems to have good performance. However to make querying for a…
NightWolf
  • 7,694
  • 9
  • 74
  • 121
10
votes
1 answer

MySQL How to Index a JSON array?

I have a JSON field with one-dimensional array. In fact, in this field I have a list of some IDs, like this: [347470, 162063, 17315, 346852, 174776, 295865, 7833, 136813] In my queries I refer this field like this: ... AND…
Red October
  • 689
  • 2
  • 12
  • 31
10
votes
3 answers

PostgreSQL union two tables and join with a third table

I want to union to tables and join them with a third metadata table and I would like to know which approach is the best/fastest? The database is a PostgreSQL. Below is my two suggestions, but other approaches are welcome. To do the join before the…
taudorf
  • 783
  • 3
  • 9
  • 27
10
votes
1 answer

Same execution path, data and schematics; different query times

We have noticed some inconsistencies in our MySQL performance for query times that we feel cannot be explained by just server load. Some queries seem to be much efficient than others, despite having a similar setup. Edit: Since opening this…
10
votes
19 answers

Optimize SQL that uses between clause

Consider the following 2 tables: Table A: id event_time Table B id start_time end_time Every record in table A is mapped to exactly 1 record in table B. This means table B has no overlapping periods. Many records from table A can be mapped to the…
daremon
  • 4,894
  • 2
  • 27
  • 27
10
votes
1 answer

Getting rid of "Using temporary; Using filesort"

When I do an explain on my query I see that it has "Using temporary; Using filesort" under "Extra" for the first row. I understand this is bad but I don't know what exactly it means or how to fix it. If you want to see my query, here's a more…
Jason Swett
  • 43,526
  • 67
  • 220
  • 351
10
votes
2 answers

How to know which count query is the fastest?

I've been exploring query optimizations in the recent releases of Spark SQL 2.3.0-SNAPSHOT and noticed different physical plans for semantically-identical queries. Let's assume I've got to count the number of rows in the following dataset: val q =…
Jacek Laskowski
  • 72,696
  • 27
  • 242
  • 420
10
votes
1 answer

Why is Postgresql using filter instead of index?

I have my postgresql ad_item table with the following columns. id | name | remaining | created_at And then I have my index on remaining which are less than or equal to 300. create index remaining_index on ad_item using btree(id) where remaining <=…
moeseth
  • 1,855
  • 5
  • 23
  • 47