Questions tagged [query-optimization]

For questions about making database queries run faster. PLEASE INCLUDE QUERY PLANS, AND TABLE AND INDEX DEFINITIONS in your question. Please use additional tags to specify your query language and server maker. Some of these questions may belong on https://DBA.stackexchange.com, especially if they involve reindexing, query plans, etc.

Please include table and index definitions for all relevant tables in your question. Please also include the query plan: the EXPLAIN output. Read on for more explanation.

If you use this is a good guide to pulling together the right information for your question.

SQL is a declarative language that can have many ways to find the same data. As a SQL programmer, you declare what you want from it, and it figures out how to get your result. Because of how databases are structured, and how they retrieve data, the same results can be retrieved in various ways that the query interpreter needs to choose between.

Using sargable predicates, using indexes, and optimizing how the query works can make it run more quickly. Restructuring the query, removing unnecessary portions of the query, or using different features of different Database Engines can all be helpful.

If the question is not (just) about how to write the query, but involves how to create new indexes, or how to optimize the server, you may get good answers on https://DBA.stackexchange.com.

Your database's query planner, also known as its query optimizer module attempts to determine the most efficient way to execute a given query by considering possible query plans.

You'll hear a lot about query planners in this tag. Each DBMS software product has its own query planner. Here's a description of one query planner.

How to ask a good query-optimization question

If you're asking "Why is my SQL query so slow?" or something like that, you need to do a little extra homework to get good answers from the Stack Overflow community. Please include this information.

What database and version?

Put this tag on your question along with the tag for your database: , . , or whatever database you use. Don't use more than one database tag, please. Optimization works somewhat differently from database to database. And, please mention the version of your database server.

How many rows and how fast?

Please figure out approximately how many rows you expect in your result set, and approximately how many rows are in each table involved with the query. Mention those numbers in your question. Please mention how long you hope your query needs to take, and how long it's taking. (That is, tell us what you mean by slow).

Don't oversimplify

Don't try to simplify your question by replacing your actual SELECT column lists with *. Using * in SELECT clauses can be harmful to query performance compared to specifying the list of columns you need.

Show your table definitions, with indexes

In your question include your table definitions and index definitions for each table and view involved in your query. Make sure your question shows all indexes in all relevant tables. When you're asking a performance question, please don't oversimplify your question by omitting indexes or columns you think might be irrelevant. The strangest things can be relevant to performance!

Very important: the names and data types of columns included in each index, with their order.

Show your database's explanation of how it satisfied your query

In your question please show the EXPLAIN PLAN output (it's just EXPLAIN or SHOWPLAN in some table server makes and models) for your query.

Things to know

  • Creating an appropriate index for a slow query is often, not always, a good solution to a problem. Answers on will often contain a suggestion for an index.

  • Avoid creating indexes on lots of single columns. Single-column indexes are often not very useful to speed up complex queries.

  • As your database grows you will, most likely, discover new slow queries. Operations can be trivial on hundreds of rows, but much more time-consuming on tens of thousands of rows. In a growing database, plan to review performance issues routinely.

  • If you use SSMS -- Microsoft's SQL Server Management Studio -- this tip is for you: Right-click in a query window, then select Show Actual Execution Plan, then run the query. The execution plan display sometimes recommends a new index to create.

Things you might do when optimizing queries

  1. Join ordering
  2. Query planning for nested SQL queries
  3. Cost estimation
  4. Parametric Query Optimization
  5. Multi-Objective Query Optimization
  6. Multi-Objective Parametric Query Optimization

Reference

Please be aware that query performance optimization is an inexact science. This is a good reference. http://use-the-index-luke.com/

7749 questions
27
votes
5 answers

Subqueries with EXISTS vs IN - MySQL

Below two queries are subqueries. Both are the same and both works fine for me. But the problem is Method 1 query takes about 10 secs to execute while Method 2 query takes under 1 sec. I was able to convert method 1 query to method 2 but I don't…
Techie
  • 44,706
  • 42
  • 157
  • 243
26
votes
2 answers

Understanding spark physical plan

I'm trying to understand physical plans on spark but I'm not understanding some parts because they seem different from traditional rdbms. For example, in this plan below, it's a plan about a query over a hive table. The query is this: select …
codin
  • 743
  • 5
  • 15
  • 27
26
votes
4 answers

How do I use DB2 Explain?

How do I use DB2's Explain function? -- both to run it, and to use it to optimize queries. Is there a better tool available for DB2? I've built queries before, but the only way I've had to tell how long they'd take is to run them and time them --…
Trevel
  • 801
  • 1
  • 7
  • 13
26
votes
7 answers

Efficiently querying a 15,000,000 rows table in MySQL

Consider the following database tables: Table "messages" with 13,000,000 rows (one row per message). Table "users" with 3,000,000 rows (one row per user). The following query is used to fetch a bunch of messages and the corresponding users: SELECT…
knorv
  • 49,059
  • 74
  • 210
  • 294
25
votes
4 answers

MySQL Join clause vs WHERE clause

What's the difference in a clause done the two following ways? SELECT * FROM table1 INNER JOIN table2 ON ( table2.col1 = table1.col2 AND table2.member_id = 4 ) I've compared them both with basic queries and EXPLAIN EXTENDED and don't see a…
Ben
  • 60,438
  • 111
  • 314
  • 488
25
votes
4 answers

PostgreSQL query runs faster with index scan, but engine chooses hash join

The query: SELECT "replays_game".* FROM "replays_game" INNER JOIN "replays_playeringame" ON "replays_game"."id" = "replays_playeringame"."game_id" WHERE "replays_playeringame"."player_id" = 50027 If I set SET enable_seqscan = off, then it does the…
24
votes
2 answers

Optimizing select with transaction under SQLite 3

I read that wrapping a lot of SELECT into BEGIN TRANSACTION/COMMIT was an interesting optimization. But are these commands really necessary if I use "PRAGMA journal_mode = OFF" before? (Which, if I remember, disables the log and obviously the…
Stef
  • 3,691
  • 6
  • 43
  • 58
24
votes
2 answers

How to interpret the output of MySQL EXPLAIN?

I want to select the content of the column text from entrytable. EXPLAIN SELECT text FROM entrytable WHERE user = 'username' && `status` = '1' && ( `status_spam_user` = 'no_spam' || ( `status_spam_user` = 'neutral' && …
R_User
  • 10,682
  • 25
  • 79
  • 120
23
votes
1 answer

mysql fix Using where;

My SQL Query: SELECT * FROM updates_cats WHERE uid =118697835834 ORDER BY created_date ASC Current Indexes: index1(uid, created_date) EXPLAIN EXTENDED result: 1 SIMPLE updates_cats ref index1 index1 8 const 2 100.00 Using where How can i fix the…
stergosz
  • 5,754
  • 13
  • 62
  • 133
23
votes
7 answers

Fastest way to find string by substring in SQL?

I have huge table with 2 columns: Id and Title. Id is bigint and I'm free to choose type of Title column: varchar, char, text, whatever. Column Title contains random text strings like "abcdefg", "q", "allyourbasebelongtous" with maximum of 255…
23
votes
1 answer

Athena: Query exhausted resources at scale factor

I am running a query like: SELECT f.*, p.countryName, p.airportName, a.name AS agentName FROM ( SELECT f.outboundlegid, f.inboundlegid, f.querydatetime, cast(f.agent as bigint) as agent, …
Jiew Meng
  • 84,767
  • 185
  • 495
  • 805
23
votes
3 answers

OR Operator Short-circuit in SQL Server

I want to consult SQL Server OR short-circuit Code: DECLARE @tempTable table ( id int ) INSERT @tempTable(id) values(1) DECLARE @id varchar(10) SET @id = 'x' SELECT * FROM @tempTable WHERE 1=1 OR id = @id --successfully SELECT…
22
votes
5 answers

How to optimize slow query with many joins

My situation: the query searches around 90,000 vehicles the query takes long each time I already have indexes on all the fields being JOINed. How can I optimise it? Here is the query: SELECT vehicles.make_id, vehicles.fuel_id, …
ChimeraTheory
  • 493
  • 1
  • 4
  • 11
22
votes
3 answers

Any point in using LIMIT in EXISTS query?

Is there any performance benefit in adding a LIMIT to an EXISTS query, or would MySQL apply the limit on its own? Example: IF EXISTS ( SELECT 1 FROM my_table LIMIT 1 -- can this improve performance? ) THEN ... END IF;
shmosel
  • 49,289
  • 6
  • 73
  • 138
22
votes
7 answers

How to Improve Query Performance with many JOINs

I have a query (with the purpose of making a view) which is using a few joins to get each column. Performance degrades quickly (exponentially?) for each set of joins added. What would be a good approach to make this query faster? Please see…
dloewen
  • 975
  • 1
  • 10
  • 26