Questions tagged [query-optimization]

For questions about making database queries run faster. PLEASE INCLUDE QUERY PLANS, AND TABLE AND INDEX DEFINITIONS in your question. Please use additional tags to specify your query language and server maker. Some of these questions may belong on https://DBA.stackexchange.com, especially if they involve reindexing, query plans, etc.

Please include table and index definitions for all relevant tables in your question. Please also include the query plan: the EXPLAIN output. Read on for more explanation.

If you use this is a good guide to pulling together the right information for your question.

SQL is a declarative language that can have many ways to find the same data. As a SQL programmer, you declare what you want from it, and it figures out how to get your result. Because of how databases are structured, and how they retrieve data, the same results can be retrieved in various ways that the query interpreter needs to choose between.

Using sargable predicates, using indexes, and optimizing how the query works can make it run more quickly. Restructuring the query, removing unnecessary portions of the query, or using different features of different Database Engines can all be helpful.

If the question is not (just) about how to write the query, but involves how to create new indexes, or how to optimize the server, you may get good answers on https://DBA.stackexchange.com.

Your database's query planner, also known as its query optimizer module attempts to determine the most efficient way to execute a given query by considering possible query plans.

You'll hear a lot about query planners in this tag. Each DBMS software product has its own query planner. Here's a description of one query planner.

How to ask a good query-optimization question

If you're asking "Why is my SQL query so slow?" or something like that, you need to do a little extra homework to get good answers from the Stack Overflow community. Please include this information.

What database and version?

Put this tag on your question along with the tag for your database: , . , or whatever database you use. Don't use more than one database tag, please. Optimization works somewhat differently from database to database. And, please mention the version of your database server.

How many rows and how fast?

Please figure out approximately how many rows you expect in your result set, and approximately how many rows are in each table involved with the query. Mention those numbers in your question. Please mention how long you hope your query needs to take, and how long it's taking. (That is, tell us what you mean by slow).

Don't oversimplify

Don't try to simplify your question by replacing your actual SELECT column lists with *. Using * in SELECT clauses can be harmful to query performance compared to specifying the list of columns you need.

Show your table definitions, with indexes

In your question include your table definitions and index definitions for each table and view involved in your query. Make sure your question shows all indexes in all relevant tables. When you're asking a performance question, please don't oversimplify your question by omitting indexes or columns you think might be irrelevant. The strangest things can be relevant to performance!

Very important: the names and data types of columns included in each index, with their order.

Show your database's explanation of how it satisfied your query

In your question please show the EXPLAIN PLAN output (it's just EXPLAIN or SHOWPLAN in some table server makes and models) for your query.

Things to know

  • Creating an appropriate index for a slow query is often, not always, a good solution to a problem. Answers on will often contain a suggestion for an index.

  • Avoid creating indexes on lots of single columns. Single-column indexes are often not very useful to speed up complex queries.

  • As your database grows you will, most likely, discover new slow queries. Operations can be trivial on hundreds of rows, but much more time-consuming on tens of thousands of rows. In a growing database, plan to review performance issues routinely.

  • If you use SSMS -- Microsoft's SQL Server Management Studio -- this tip is for you: Right-click in a query window, then select Show Actual Execution Plan, then run the query. The execution plan display sometimes recommends a new index to create.

Things you might do when optimizing queries

  1. Join ordering
  2. Query planning for nested SQL queries
  3. Cost estimation
  4. Parametric Query Optimization
  5. Multi-Objective Query Optimization
  6. Multi-Objective Parametric Query Optimization

Reference

Please be aware that query performance optimization is an inexact science. This is a good reference. http://use-the-index-luke.com/

7749 questions
2
votes
3 answers

Sql Query optimization Issue in JOIN

I have multiple JOIN statements in one query and I have to check two settings in the JOIN. How to do that in best way? table: WebSettings ------------------------------------------------------- Name | Setting | InstID |…
Keppy
  • 471
  • 1
  • 7
  • 23
2
votes
3 answers

DB Table Optimization join vs repeat columns

This is more of a preference but I was wondering what people think would be the optimal option to perform. I have a Question, Answer, and Point (because I need to track which user made the point) Table Dump Question: id title Answer: id …
Atherion
  • 584
  • 1
  • 5
  • 14
2
votes
2 answers

MySQL: Is it better to have an index and a small filesort or no index and no filesort?

I have a large database (180k+ rows and growing fast) of location data and am plotting them on a google map. For a given view port, I just want to serve up a sample of 100 applicable points. The database is therefore queried by lat/lng, but if I…
user435281
  • 79
  • 2
  • 6
2
votes
3 answers

SQL Server 2008 update table and switch values query optimization

I have a table with values ID Son Father ----------- ---------- ---------- 1 Mark Gerard 2 Gerard Ivan 3 Leo Samuel 4 Samuel Johan 5 Ivan Carles I need to…
Viktor Krykun
  • 243
  • 1
  • 2
  • 9
2
votes
1 answer

Any way to optimize this MySQL query? (Resource intense)

My app needs to run this query pretty often, which gets a list of user data for the app to display. The problem is that subquery about the user_quiz is resource heavy and calculating the rankings are also very CPU intense too. Benchmark: ~.5 second…
Yinan Fang
  • 265
  • 4
  • 11
2
votes
2 answers

Speed up Django & Postgres with simple JSON field

I have a very very complex model with lots of related models by FK and M2M which are also have lots of relations, etc. So, rendering a list of such objects is a very expensive SQL operation, and i want to optimise it. (select_related and…
MaxCore
  • 2,438
  • 4
  • 25
  • 43
2
votes
2 answers

What affects amount of data shuffled in spark

For example Im executing some queries on spark, and in the spark UI I can see that some queries have more shuffle , and this shuffle seems that is the amount of data read locally and read between executors. But so Im not understanding one thing, for…
jUsr
  • 301
  • 1
  • 4
  • 9
2
votes
1 answer

Query same table multiple times

I have always used multiple joins when I need to reference same data on different columns. Like this: select a.ContactName ,b.ContactName ,c.ContactName from OrderBase as o left join ContactBase as a on o.contactid1 =…
Sweetspot
  • 91
  • 1
  • 9
2
votes
1 answer

Better to use a document with a lot of objects or just a lot of documents?

So I have a little bit of data on a lot of people in a company such as their names, age, and gender. I'm going to store their info in MongoDB. Would it be better for me to store their info in a lot of documents or in one document as a bunch of…
Joe Thomas
  • 5,807
  • 6
  • 25
  • 36
2
votes
2 answers

hive left outer join long running

Hortonworks HDP 2.3.0 - Hive 0.14 Table T1 ( partition on col1, no bucket, ORC ) app 120 million rows & 6GB datasize Table T2 ( partition on col2, no bucket, ORC ) app 200 M rows & 6MB datasize T1 left outer join on t2 ( t1.col3 = t2.col3 ) The…
Venkat M
  • 21
  • 2
2
votes
1 answer

What to prefer in query optimization: Using filesort or more rows examined

I'm trying to optimize this mysql query using EXPLAIN. Can somebody please help me out over here? EXPLAIN SELECT * FROM keyword WHERE keyword LIKE "panasonic%" AND keyword != "panasonic" AND price < 3230 AND price > 3370 ORDER BY price DESC LIMIT…
2
votes
3 answers

Is it true that writing the database name in every query is faster than calling mysql_select_db() on every page load?

I work at a fairly big website; we have ~400-500 million page views a month. We use PHP and MySQL. Currently our page flow works like this (semi pseudo code for clarity): mysql_connect(); mysql_select_db('red'); mysql_query('SELECT * FROM…
anon
  • 23
  • 2
2
votes
4 answers

Query Optimization for large database

Hi i am in need of help to optimize a query for large database records above 1 Millions . Current query is taking 27-30 seconds to execute. SELECT SQL_CALC_FOUND_ROWS candidate.candidate_id AS candidateID, candidate.candidate_id AS…
Vipin Singh
  • 532
  • 1
  • 7
  • 24
2
votes
1 answer

SQL averaging multiple time periods of the same dataset

I have the following query: SELECT AVG(val) from floatTable WHERE tagindex IN(1,2,3,4) AND DateAndTime > '$first_of_year' It returns the average value for all the values measured for those four tags for the year to date. Since I'm already…
user438199
  • 63
  • 5
2
votes
3 answers

How can my SQL query be optimized?

I have a problem with a quite slowish MYSQL query. I'm building an AJAX menu with PHP and performance is really an issue. The query takes about 0,5 sec to complete, and i don't know how to optimize it. The SQL query : SELECT M.nom_mat, M.id_mat,…