Questions tagged [query-optimization]

For questions about making database queries run faster. PLEASE INCLUDE QUERY PLANS, AND TABLE AND INDEX DEFINITIONS in your question. Please use additional tags to specify your query language and server maker. Some of these questions may belong on https://DBA.stackexchange.com, especially if they involve reindexing, query plans, etc.

Please include table and index definitions for all relevant tables in your question. Please also include the query plan: the EXPLAIN output. Read on for more explanation.

If you use this is a good guide to pulling together the right information for your question.

SQL is a declarative language that can have many ways to find the same data. As a SQL programmer, you declare what you want from it, and it figures out how to get your result. Because of how databases are structured, and how they retrieve data, the same results can be retrieved in various ways that the query interpreter needs to choose between.

Using sargable predicates, using indexes, and optimizing how the query works can make it run more quickly. Restructuring the query, removing unnecessary portions of the query, or using different features of different Database Engines can all be helpful.

If the question is not (just) about how to write the query, but involves how to create new indexes, or how to optimize the server, you may get good answers on https://DBA.stackexchange.com.

Your database's query planner, also known as its query optimizer module attempts to determine the most efficient way to execute a given query by considering possible query plans.

You'll hear a lot about query planners in this tag. Each DBMS software product has its own query planner. Here's a description of one query planner.

How to ask a good query-optimization question

If you're asking "Why is my SQL query so slow?" or something like that, you need to do a little extra homework to get good answers from the Stack Overflow community. Please include this information.

What database and version?

Put this tag on your question along with the tag for your database: , . , or whatever database you use. Don't use more than one database tag, please. Optimization works somewhat differently from database to database. And, please mention the version of your database server.

How many rows and how fast?

Please figure out approximately how many rows you expect in your result set, and approximately how many rows are in each table involved with the query. Mention those numbers in your question. Please mention how long you hope your query needs to take, and how long it's taking. (That is, tell us what you mean by slow).

Don't oversimplify

Don't try to simplify your question by replacing your actual SELECT column lists with *. Using * in SELECT clauses can be harmful to query performance compared to specifying the list of columns you need.

Show your table definitions, with indexes

In your question include your table definitions and index definitions for each table and view involved in your query. Make sure your question shows all indexes in all relevant tables. When you're asking a performance question, please don't oversimplify your question by omitting indexes or columns you think might be irrelevant. The strangest things can be relevant to performance!

Very important: the names and data types of columns included in each index, with their order.

Show your database's explanation of how it satisfied your query

In your question please show the EXPLAIN PLAN output (it's just EXPLAIN or SHOWPLAN in some table server makes and models) for your query.

Things to know

  • Creating an appropriate index for a slow query is often, not always, a good solution to a problem. Answers on will often contain a suggestion for an index.

  • Avoid creating indexes on lots of single columns. Single-column indexes are often not very useful to speed up complex queries.

  • As your database grows you will, most likely, discover new slow queries. Operations can be trivial on hundreds of rows, but much more time-consuming on tens of thousands of rows. In a growing database, plan to review performance issues routinely.

  • If you use SSMS -- Microsoft's SQL Server Management Studio -- this tip is for you: Right-click in a query window, then select Show Actual Execution Plan, then run the query. The execution plan display sometimes recommends a new index to create.

Things you might do when optimizing queries

  1. Join ordering
  2. Query planning for nested SQL queries
  3. Cost estimation
  4. Parametric Query Optimization
  5. Multi-Objective Query Optimization
  6. Multi-Objective Parametric Query Optimization

Reference

Please be aware that query performance optimization is an inexact science. This is a good reference. http://use-the-index-luke.com/

7749 questions
2
votes
2 answers

Comparision operator with in properies of query of Neo4j

I am in need of get data of whether there is no relation exists between two labels and condition based data on one of labels. I found an answer following :: MATCH (n:Label1) WHERE NOT (n)-[:REL_1]-(:Label2) OR (n)-[:REL_1]-(e:Label2 {id:1})…
chikku
  • 863
  • 1
  • 7
  • 19
2
votes
5 answers

Optimize SQL to get rows count

I have a page on my site that keeps track of the number of people accessing it, on another part I displays the data containing information about the users that access this page, it displays only about 10 at a time. The problem is I need to create…
George
  • 3,757
  • 9
  • 51
  • 86
2
votes
1 answer

How can I optimize this really slow query generated by Django?

here's my Django ORM query: Group.objects.filter(public = True)\ .annotate(num_members = Count('members', distinct = True))\ .annotate(num_images = Count('images', distinct = True))\ .order_by(sort) Unfortunately this is taking over 30…
Salvatore Iovene
  • 2,064
  • 1
  • 17
  • 31
2
votes
1 answer

Is there a way to map array of _id's to documents in one query using MongoDB?

I have a MongoDB database and 2 collections inside it: programs and exercises. Documents inside programs collection have an exercises field that contains a string representation of IDs in exercises table, like that: { "_id" :…
serge1peshcoff
  • 4,342
  • 11
  • 45
  • 76
2
votes
1 answer

Why is a query with indexes not a covered query?

You would like to perform a covered query on the example collection. You have the following indexes: { name : 1, dob : 1 } { _id : 1 } { hair : 1, name : 1 } Why is the below query not a covered query? db.example.find( { name : { $in : [ "Bart",…
Zameer Ansari
  • 28,977
  • 24
  • 140
  • 219
2
votes
0 answers

Optimizing time and memory for fetching large number of objects in django?

There are multiple tables containing more than 1 million records, and I have to do something like : for record in GiantTable.objects.all(): doSomethingAwesome(record) I have tried: Django paginator, but again it does multiple calls to…
2
votes
1 answer

Stored procedure with FAST_FORWARD cursor loop starts fast, ends slow

I have a stored procedure that uses a FAST_FORWARD cursor to chronologically loop over a set of ~300k records and assigns them to declaration sets based on the state of a lot of running variables and flags, partly implemented as table variables. I…
littlegreen
  • 7,290
  • 9
  • 45
  • 51
2
votes
1 answer

MySQL: Optimization GROUP BY multiple keys

I have a table PAYMENTS in MySql database: CREATE TABLE `PAYMENTS` ( `ID` BIGINT(20) NOT NULL AUTO_INCREMENT, `USER_ID` BIGINT(20) NOT NULL, `CATEGORY_ID` BIGINT(20) NOT NULL, `AMOUNT` DOUBLE NULL DEFAULT NULL, PRIMARY KEY…
semitkin
  • 53
  • 1
  • 6
2
votes
3 answers

Optimize joins from multiple tables

How can I optimize Performance of the below mentioned query when the table structure is as shown in the pic below Pic Showing The Table Structure select CounterID, OutletTitle, CounterTitle from( select OutletID, Text as OutletTitle from…
2
votes
2 answers

Is it worth to create a PreparedStatement for int values?

I create PreparedStatement when I need to pass arguments to the answers there do not address your problem, please edit to explain in detail the parts of your question that are unique. Title Is it worth to create a PreparedStatement for int values? …
2
votes
2 answers

Does MySQL use existing indexes on creating new indexes?

I have a large table with millions of records. Table `price` ------------ id product site value The table is brand new, and there are no indexes created. I then issued a request for new index creation with the following query: CREATE INDEX…
Pentium10
  • 204,586
  • 122
  • 423
  • 502
2
votes
1 answer

Optimizing AND conditions in postgresql

I am writing analytics software that generates complicated queries. When building the where clause, it may happen that multiple constraints are added on the same database field. I was wondering if PostgreSQL rewrites multiple constraints into…
2
votes
1 answer

Optimize mongoDB aggregation that uses $min and $max

I have about 3 million documents in my database. I have this query to get the min and max latitudes and longitudes contains in my documents because in my app I want to zoom on the existing data contained within a square. It takes about 16 seconds…
SwissFr
  • 178
  • 1
  • 12
2
votes
1 answer

Keep the result of subset from $match aggregation in cache in mongoDB

I am doing a website to explore mongoDB data. In my database I store GPS measurements captured from smartphones. I am using various queries to explore those measurements. I have one query that groups by day and count the measurements. Another query…
SwissFr
  • 178
  • 1
  • 12
2
votes
4 answers

Aggregate to 'plain' query

I have a query which uses aggregate functions to assign the maximum absolute of the values to another column in the table. The problem is that it takes whole lot of time (apprx. adds upto 10-15 seconds) to query completion time. This is what the…
faizanjehangir
  • 2,771
  • 6
  • 45
  • 83