0

I asked the wrong question so I will try again (sorry). You don't have to read the query in details, but just to give you an idea.

Anyway: the query in question is this, generated by Doctrine2 (I cut SELECT part)

FROM
    tbl_story t0_ 
      LEFT JOIN tbl_user t1_ ON t0_.creator_id = t1_.id 
      LEFT JOIN tbl_image t2_  ON t1_.avatar_id = t2_.id 
      LEFT JOIN tbl_image t3_  ON t0_.id = t3_.story_id 
      LEFT JOIN tbl_like t4_ ON t0_.id = t4_.story_id AND (t4_.creator_id = 3) 
      LEFT JOIN tbl_comment t5_ ON t0_.id = t5_.story_id 
      LEFT JOIN tbl_like t6_  ON t5_.id = t6_.comment_id AND (t6_.creator_id = 3) 
      LEFT JOIN tbl_user t7_  ON t5_.creator_id = t7_.id 
      LEFT JOIN tbl_image t8_ ON t7_.avatar_id = t8_.id 
      LEFT JOIN tbl_location_city t9_  ON t0_.city_id = t9_.id 
      LEFT JOIN tbl_location_state t10_ ON t9_.state_id = t10_.id 
      LEFT JOIN tbl_location_country t11_ ON t10_.country_id = t11_.id 
      LEFT JOIN tbl_story t12_ ON t0_.parent_id = t12_.id 
      LEFT JOIN tbl_page t13_ ON t0_.page_id = t13_.id 
      LEFT JOIN tbl_page_type t14_ ON t13_.page_type_id = t14_.id 
      LEFT JOIN tbl_story_tag t25_  ON t0_.id = t25_.story_id 
      LEFT JOIN tbl_tag t15_ ON t15_.id = t25_.tag_id 
      LEFT JOIN tbl_story_linked_user_reference t16_ ON t0_.id = t16_.story_id 
      LEFT JOIN tbl_user t17_ ON t16_.user_id = t17_.id 
      LEFT JOIN tbl_image t18_ ON t17_.avatar_id = t18_.id 
      LEFT JOIN tbl_story t19_  ON t0_.id = t19_.parent_id 
      LEFT JOIN tbl_album t20_ ON t0_.parent_album_id = t20_.id 
      LEFT JOIN tbl_image t21_ ON t20_.id = t21_.album_id 
      LEFT JOIN tbl_image_linked_user t22_ ON t21_.id = t22_.image_id 
      LEFT JOIN tbl_user t23_ ON t22_.user_id = t23_.id 
      LEFT JOIN tbl_image t24_ ON t23_.avatar_id = t24_.id 
WHERE (
    t0_.creator_id = 3 
    OR t0_.id IN 
        (SELECT 
          t26_.id 
        FROM tbl_story t26_ 
          INNER JOIN tbl_story_circle_visibility t28_ ON t26_.id = t28_.story_id 
          INNER JOIN tbl_circle t27_ ON t27_.id = t28_.circle_id 
          INNER JOIN tbl_user_circle_reference t29_ ON t27_.id = t29_.circle_id 
            AND (t29_.user_id = 3))
    OR t0_.is_public = 1
  ) 
  AND t0_.is_draft = 0 
  AND t0_.type IN (0, 3, 2, 21) 
ORDER BY t0_.created_at DESC
LIMIT 7 

The problem: with ORDER BY, query takes 1.2 seconds to execute when there are only 31.000 rows in tbl_story (main query). If I remove ORDER BY, it is executed in 6ms, I guess all these joins take extra time but still nothing to worry.

I first thought ORDER BY is slow but I was wrong. I tried

SELECT * FROM `tbl_story` ORDER BY created_at DESC LIMIT 7

(same thing but without joins) and it is executed in 2ms.

I read other questions too including this one that suggests removing JOIN parts. That might work if I this was possible in Doctrine2:

$queryBuilder->select("o.id")->removeLeftJoins()

and after I fetch ID's, use them in clone of original query.

How can I solver the problem of pagination?

NOTE:

Even if I completelly remove WHERE clause, the query takes same 1.2 secs so it is not the one giving problems.

Thanks.

Community
  • 1
  • 1
Zeljko
  • 5,048
  • 5
  • 36
  • 46
  • I was sad since morning but You made my day ROFL :D – Moeed Farooqui Jul 04 '13 at 12:59
  • Can you post and explain for the query? I suspect that some of the tables you are joined and not using indexed columns – Kickstart Jul 04 '13 at 12:59
  • http://pastebin.com/6evBkJF3 It is kinda hard to read because it has 30 rows. Doctrine2 automatically creates indexes when it finds any type of relation. – Zeljko Jul 04 '13 at 13:15
  • @MoeedFarooqui I don't what is funny in my question. If you know the answer, please, enlighten me :) – Zeljko Jul 04 '13 at 13:22
  • 1
    It has used no index on tbl_story (t0_), and also seems to have missed tbl_image_linked_user (t22_). For tbl_story, add an index on type AND is_draft AND created_at, although that might not help much. To go further you might need to split it into 3 queries unioned together to allow it to use useful indexes. – Kickstart Jul 04 '13 at 13:25
  • Yep, I forgot them. But I tried to order by id (which is indexed), added indexes you told but no speed difference. Anyway, thanks for trying I gave you thumbs up. – Zeljko Jul 04 '13 at 13:33

1 Answers1

0

I would suspect that indexing on type and is_draft might not narrow things down enough for MySQL to take any notice of those indexes.

If you have an index on creator_id, another on is_public and another on id (which I presume you do as it is probably the primary id) then 3 unioned queries like this might do some good.

SELECT *
FROM
    tbl_story t0_ 
      LEFT JOIN tbl_user t1_ ON t0_.creator_id = t1_.id 
      LEFT JOIN tbl_image t2_  ON t1_.avatar_id = t2_.id 
      LEFT JOIN tbl_image t3_  ON t0_.id = t3_.story_id 
      LEFT JOIN tbl_like t4_ ON t0_.id = t4_.story_id AND (t4_.creator_id = 3) 
      LEFT JOIN tbl_comment t5_ ON t0_.id = t5_.story_id 
      LEFT JOIN tbl_like t6_  ON t5_.id = t6_.comment_id AND (t6_.creator_id = 3) 
      LEFT JOIN tbl_user t7_  ON t5_.creator_id = t7_.id 
      LEFT JOIN tbl_image t8_ ON t7_.avatar_id = t8_.id 
      LEFT JOIN tbl_location_city t9_  ON t0_.city_id = t9_.id 
      LEFT JOIN tbl_location_state t10_ ON t9_.state_id = t10_.id 
      LEFT JOIN tbl_location_country t11_ ON t10_.country_id = t11_.id 
      LEFT JOIN tbl_story t12_ ON t0_.parent_id = t12_.id 
      LEFT JOIN tbl_page t13_ ON t0_.page_id = t13_.id 
      LEFT JOIN tbl_page_type t14_ ON t13_.page_type_id = t14_.id 
      LEFT JOIN tbl_story_tag t25_  ON t0_.id = t25_.story_id 
      LEFT JOIN tbl_tag t15_ ON t15_.id = t25_.tag_id 
      LEFT JOIN tbl_story_linked_user_reference t16_ ON t0_.id = t16_.story_id 
      LEFT JOIN tbl_user t17_ ON t16_.user_id = t17_.id 
      LEFT JOIN tbl_image t18_ ON t17_.avatar_id = t18_.id 
      LEFT JOIN tbl_story t19_  ON t0_.id = t19_.parent_id 
      LEFT JOIN tbl_album t20_ ON t0_.parent_album_id = t20_.id 
      LEFT JOIN tbl_image t21_ ON t20_.id = t21_.album_id 
      LEFT JOIN tbl_image_linked_user t22_ ON t21_.id = t22_.image_id 
      LEFT JOIN tbl_user t23_ ON t22_.user_id = t23_.id 
      LEFT JOIN tbl_image t24_ ON t23_.avatar_id = t24_.id 
WHERE t0_.creator_id = 3 
  AND t0_.is_draft = 0 
  AND t0_.type IN (0, 3, 2, 21) 
UNION
SELECT *
FROM
    tbl_story t0_ 
      LEFT JOIN tbl_user t1_ ON t0_.creator_id = t1_.id 
      LEFT JOIN tbl_image t2_  ON t1_.avatar_id = t2_.id 
      LEFT JOIN tbl_image t3_  ON t0_.id = t3_.story_id 
      LEFT JOIN tbl_like t4_ ON t0_.id = t4_.story_id AND (t4_.creator_id = 3) 
      LEFT JOIN tbl_comment t5_ ON t0_.id = t5_.story_id 
      LEFT JOIN tbl_like t6_  ON t5_.id = t6_.comment_id AND (t6_.creator_id = 3) 
      LEFT JOIN tbl_user t7_  ON t5_.creator_id = t7_.id 
      LEFT JOIN tbl_image t8_ ON t7_.avatar_id = t8_.id 
      LEFT JOIN tbl_location_city t9_  ON t0_.city_id = t9_.id 
      LEFT JOIN tbl_location_state t10_ ON t9_.state_id = t10_.id 
      LEFT JOIN tbl_location_country t11_ ON t10_.country_id = t11_.id 
      LEFT JOIN tbl_story t12_ ON t0_.parent_id = t12_.id 
      LEFT JOIN tbl_page t13_ ON t0_.page_id = t13_.id 
      LEFT JOIN tbl_page_type t14_ ON t13_.page_type_id = t14_.id 
      LEFT JOIN tbl_story_tag t25_  ON t0_.id = t25_.story_id 
      LEFT JOIN tbl_tag t15_ ON t15_.id = t25_.tag_id 
      LEFT JOIN tbl_story_linked_user_reference t16_ ON t0_.id = t16_.story_id 
      LEFT JOIN tbl_user t17_ ON t16_.user_id = t17_.id 
      LEFT JOIN tbl_image t18_ ON t17_.avatar_id = t18_.id 
      LEFT JOIN tbl_story t19_  ON t0_.id = t19_.parent_id 
      LEFT JOIN tbl_album t20_ ON t0_.parent_album_id = t20_.id 
      LEFT JOIN tbl_image t21_ ON t20_.id = t21_.album_id 
      LEFT JOIN tbl_image_linked_user t22_ ON t21_.id = t22_.image_id 
      LEFT JOIN tbl_user t23_ ON t22_.user_id = t23_.id 
      LEFT JOIN tbl_image t24_ ON t23_.avatar_id = t24_.id 
WHERE t0_.id IN 
        (SELECT 
          t26_.id 
        FROM tbl_story t26_ 
          INNER JOIN tbl_story_circle_visibility t28_ ON t26_.id = t28_.story_id 
          INNER JOIN tbl_circle t27_ ON t27_.id = t28_.circle_id 
          INNER JOIN tbl_user_circle_reference t29_ ON t27_.id = t29_.circle_id 
            AND (t29_.user_id = 3))
  AND t0_.is_draft = 0 
  AND t0_.type IN (0, 3, 2, 21) 
UNION
SELECT *
FROM
    tbl_story t0_ 
      LEFT JOIN tbl_user t1_ ON t0_.creator_id = t1_.id 
      LEFT JOIN tbl_image t2_  ON t1_.avatar_id = t2_.id 
      LEFT JOIN tbl_image t3_  ON t0_.id = t3_.story_id 
      LEFT JOIN tbl_like t4_ ON t0_.id = t4_.story_id AND (t4_.creator_id = 3) 
      LEFT JOIN tbl_comment t5_ ON t0_.id = t5_.story_id 
      LEFT JOIN tbl_like t6_  ON t5_.id = t6_.comment_id AND (t6_.creator_id = 3) 
      LEFT JOIN tbl_user t7_  ON t5_.creator_id = t7_.id 
      LEFT JOIN tbl_image t8_ ON t7_.avatar_id = t8_.id 
      LEFT JOIN tbl_location_city t9_  ON t0_.city_id = t9_.id 
      LEFT JOIN tbl_location_state t10_ ON t9_.state_id = t10_.id 
      LEFT JOIN tbl_location_country t11_ ON t10_.country_id = t11_.id 
      LEFT JOIN tbl_story t12_ ON t0_.parent_id = t12_.id 
      LEFT JOIN tbl_page t13_ ON t0_.page_id = t13_.id 
      LEFT JOIN tbl_page_type t14_ ON t13_.page_type_id = t14_.id 
      LEFT JOIN tbl_story_tag t25_  ON t0_.id = t25_.story_id 
      LEFT JOIN tbl_tag t15_ ON t15_.id = t25_.tag_id 
      LEFT JOIN tbl_story_linked_user_reference t16_ ON t0_.id = t16_.story_id 
      LEFT JOIN tbl_user t17_ ON t16_.user_id = t17_.id 
      LEFT JOIN tbl_image t18_ ON t17_.avatar_id = t18_.id 
      LEFT JOIN tbl_story t19_  ON t0_.id = t19_.parent_id 
      LEFT JOIN tbl_album t20_ ON t0_.parent_album_id = t20_.id 
      LEFT JOIN tbl_image t21_ ON t20_.id = t21_.album_id 
      LEFT JOIN tbl_image_linked_user t22_ ON t21_.id = t22_.image_id 
      LEFT JOIN tbl_user t23_ ON t22_.user_id = t23_.id 
      LEFT JOIN tbl_image t24_ ON t23_.avatar_id = t24_.id 
WHERE t0_.is_public = 1
  AND t0_.is_draft = 0 
  AND t0_.type IN (0, 3, 2, 21) 
ORDER BY t0_.created_at DESC
LIMIT 7 
Kickstart
  • 21,403
  • 2
  • 21
  • 33