Questions tagged [query-optimization]

For questions about making database queries run faster. PLEASE INCLUDE QUERY PLANS, AND TABLE AND INDEX DEFINITIONS in your question. Please use additional tags to specify your query language and server maker. Some of these questions may belong on https://DBA.stackexchange.com, especially if they involve reindexing, query plans, etc.

Please include table and index definitions for all relevant tables in your question. Please also include the query plan: the EXPLAIN output. Read on for more explanation.

If you use this is a good guide to pulling together the right information for your question.

SQL is a declarative language that can have many ways to find the same data. As a SQL programmer, you declare what you want from it, and it figures out how to get your result. Because of how databases are structured, and how they retrieve data, the same results can be retrieved in various ways that the query interpreter needs to choose between.

Using sargable predicates, using indexes, and optimizing how the query works can make it run more quickly. Restructuring the query, removing unnecessary portions of the query, or using different features of different Database Engines can all be helpful.

If the question is not (just) about how to write the query, but involves how to create new indexes, or how to optimize the server, you may get good answers on https://DBA.stackexchange.com.

Your database's query planner, also known as its query optimizer module attempts to determine the most efficient way to execute a given query by considering possible query plans.

You'll hear a lot about query planners in this tag. Each DBMS software product has its own query planner. Here's a description of one query planner.

How to ask a good query-optimization question

If you're asking "Why is my SQL query so slow?" or something like that, you need to do a little extra homework to get good answers from the Stack Overflow community. Please include this information.

What database and version?

Put this tag on your question along with the tag for your database: , . , or whatever database you use. Don't use more than one database tag, please. Optimization works somewhat differently from database to database. And, please mention the version of your database server.

How many rows and how fast?

Please figure out approximately how many rows you expect in your result set, and approximately how many rows are in each table involved with the query. Mention those numbers in your question. Please mention how long you hope your query needs to take, and how long it's taking. (That is, tell us what you mean by slow).

Don't oversimplify

Don't try to simplify your question by replacing your actual SELECT column lists with *. Using * in SELECT clauses can be harmful to query performance compared to specifying the list of columns you need.

Show your table definitions, with indexes

In your question include your table definitions and index definitions for each table and view involved in your query. Make sure your question shows all indexes in all relevant tables. When you're asking a performance question, please don't oversimplify your question by omitting indexes or columns you think might be irrelevant. The strangest things can be relevant to performance!

Very important: the names and data types of columns included in each index, with their order.

Show your database's explanation of how it satisfied your query

In your question please show the EXPLAIN PLAN output (it's just EXPLAIN or SHOWPLAN in some table server makes and models) for your query.

Things to know

  • Creating an appropriate index for a slow query is often, not always, a good solution to a problem. Answers on will often contain a suggestion for an index.

  • Avoid creating indexes on lots of single columns. Single-column indexes are often not very useful to speed up complex queries.

  • As your database grows you will, most likely, discover new slow queries. Operations can be trivial on hundreds of rows, but much more time-consuming on tens of thousands of rows. In a growing database, plan to review performance issues routinely.

  • If you use SSMS -- Microsoft's SQL Server Management Studio -- this tip is for you: Right-click in a query window, then select Show Actual Execution Plan, then run the query. The execution plan display sometimes recommends a new index to create.

Things you might do when optimizing queries

  1. Join ordering
  2. Query planning for nested SQL queries
  3. Cost estimation
  4. Parametric Query Optimization
  5. Multi-Objective Query Optimization
  6. Multi-Objective Parametric Query Optimization

Reference

Please be aware that query performance optimization is an inexact science. This is a good reference. http://use-the-index-luke.com/

7749 questions
14
votes
12 answers

Optimizing MySQL search query

Need yours help optimizing one mysql query. Lets take simple table for example. CREATE TABLE `Modules` ( `ID` int(11) NOT NULL AUTO_INCREMENT, `moduleName` varchar(100) NOT NULL, `menuName` varchar(255) NOT NULL, PRIMARY KEY (`ID`), KEY…
Denis O.
  • 1,841
  • 19
  • 37
13
votes
5 answers

Why are UNION queries so slow in MySQL?

When I optimize my 2 single queries to run in less than 0.02 seconds and then UNION them the resulting query takes over 1 second to run. Also, a UNION ALL takes longer than a UNION DISTINCT. I would assume allowing duplicates would make the query…
Greg
  • 7,233
  • 12
  • 42
  • 53
13
votes
3 answers

MySQL: Why does an Order By ID runs much slower than Order By other Columns?

I am using MySQL version 5.5.14 to run the following query, QUERY 1, from a table of 5 Million rows: SELECT P.ID, P.Type, P.Name, P.cty , X(P.latlng) as 'lat', Y(P.latlng) as 'lng' , P.cur, P.ak, P.tn, P.St, P.Tm, P.flA, P.ldA, P.flN …
Question Overflow
  • 10,925
  • 18
  • 72
  • 110
13
votes
7 answers

How to search millions of record in SQL table faster?

I have SQL table with millions of domain name. But now when I search for let's say SELECT * FROM tblDomainResults WHERE domainName LIKE '%lifeis%' It takes more than 10 minutes to get the results. I tried indexing but that didn't help. What…
user737063
  • 139
  • 1
  • 1
  • 3
13
votes
1 answer

Higher cardinality column first in an index when involving a range?

CREATE TABLE `files` ( `did` int(10) unsigned NOT NULL DEFAULT '0', `filename` varbinary(200) NOT NULL, `ext` varbinary(5) DEFAULT NULL, `fsize` double DEFAULT NULL, `filetime` datetime DEFAULT NULL, PRIMARY KEY (`did`,`filename`), KEY…
Rick James
  • 135,179
  • 13
  • 127
  • 222
13
votes
3 answers

I'm not sure if I have the correct indexes or if I can improve the speed of my query in MySQL?

My query has a join, and it looks like it's using two indexes which makes it more complicated. I'm not sure if I can improve on this, but I thought I'd ask. The query produces a list of records with similar keywords the record being queried. Here's…
Jules
  • 7,568
  • 14
  • 102
  • 186
13
votes
3 answers

How can I learn to optimize SQL queries

I know how to write SQL queries and can get the results I need. But sometimes my queries are slow and I don't fully understand why. What are some good resources for learning to write efficient queries and to optimize the queries I've already…
epotter
  • 7,631
  • 7
  • 63
  • 88
13
votes
10 answers

When designing databases, what is the preferred way to store multiple true / false values?

As stated in the title, when designing databases, what is the preferred way to handle tables that have multiple columns that are just storing true / false values as just a single either or value (e.g. "Y/N: or "0/1")? Likewise, are there some issues…
rjzii
  • 14,236
  • 12
  • 79
  • 119
13
votes
4 answers

Techniques for reducing database queries in a Rails app

If you have a Rail app with many complex associated models, what techniques do you employ to reduce database queries? In fact, I'll extend that question a little further and ask, what do you consider "too many" queries for any page? I have a page…
13
votes
4 answers

Is there an "Explain Query" for MongoDB Linq?

Is there a way to run .explain() or equivalent on Linq queries? I would want to know The text of the actual JSON query The output of .explain() (indexes used, etc) It would also be nice to have the execution time of the query
kelloti
  • 8,705
  • 5
  • 46
  • 82
13
votes
6 answers

Optimising MySQL queries across hierarchical data

I have a fairly stable directed graph of order ~100k vertices and size ~1k edges. It is two-dimensional insofar as its vertices can be identified by a pair of integers (x, y) (of cardinality ~100 x ~1000) and all edges are strictly increasing in…
eggyal
  • 122,705
  • 18
  • 212
  • 237
12
votes
2 answers

How do I limit the number of results for a specific variable in a SPARQL query?

Let's say I have a SPARQL query like this, looking for resources that have some shared property with a focal resource, and also getting some other statements about the focal resource : CONSTRUCT { ?focal pred:icate ?shared . ?other pred:icate…
12
votes
3 answers

SQL question from Joel Spolsky article

From Joel Spolsky's article on leaky abstractions: [C]ertain SQL queries are thousands of times slower than other logically equivalent queries. A famous example of this is that some SQL servers are dramatically faster if you specify "where a=b and…
jason
  • 236,483
  • 35
  • 423
  • 525
12
votes
5 answers

Alternative to COUNT for innodb to prevent table scan?

I've managed to put together a query that works for my needs, albeit more complicated than I was hoping. But, for the size of tables the query is slower than it should be (0.17s). The reason, based on the EXPLAIN provided below, is because there is…
EternalHour
  • 8,308
  • 6
  • 38
  • 57
12
votes
3 answers

MySQL, delete and index hint

I have to delete about 10K rows from a table that has more than 100 million rows based on some criteria. When I execute the query, it takes about 5 minutes. I ran an explain plan (the delete query converted to select * since MySQL does not support…
Manuel Darveau
  • 4,585
  • 5
  • 26
  • 36