0

I have a large table named 'roomlogs' which has nearly 1 million entries. The structure of the table:

id --> PK

roomId --> varchar FK to rooms table

userId --> varchar FK to users table

enterTime --> Date and Time

exitTime --> Date and Time

status --> bool

I have the previous indexing on roomID, I recently added an index on the userId column.

So, When I run a stored procedure with following code it is taking more time like on average 50 seconds. WHich it should not take.

    DELIMITER ;;
    CREATE DEFINER=`root`@`%` PROCEDURE `enter_room`(IN pRoomId varchar(200), IN puserId varchar(50), IN ptime datetime, IN phidden int, pcheckid int, pexit datetime)

begin
    update roomlogs set 
          roomlogs.exitTime = ptime,
          roomlogs.`status` = 1  
       where 
              roomlogs.userId = puserId 
          and roomlogs.`status` = 0 
          and DATEDIFF(ptime,roomlogs.enterTime) = 0;

    INSERT into roomlogs 
      ( roomlogs.roomId,
        roomlogs.userId,
        roomlogs.enterTime,
        roomlogs.exitTime,
        roomlogs.hidden,
        roomlogs.checkinId ) 
      value
      ( pRoomId,
        userId,
        ptime,
        pexit,
        phidden,
        pcheckid);

    select * 
       from 
          roomlogs 
       where 
          roomlogs.id= LAST_INSERT_ID();
    end ;;
    DELIMITER ;

What Can be the reason for it to take this much time:

  1. I added an index recently so previous rows are not indexed.
  2. There is no selection on storage type for any indexes right now. Should I change it to B-tree?
  3. On my website, I get 20-30 simultaneous call on other procedures also while this procedure has 10-20 simultaneous calls, does the update query in the procedure make a lock? But in MySQL.slow_logs table for each query the lock _time shows 0.
  4. Is there any other reason for this behaviour?

Edit: Here is the SHOW TABLE:

CREATE TABLE `roomlogs` (
   `roomId` varchar(200) CHARACTER SET latin1 DEFAULT NULL,
   `userID` varchar(50) CHARACTER SET latin1 DEFAULT NULL,
   `enterTime` datetime DEFAULT NULL,
   `exitTime` datetime DEFAULT NULL,
   `id` int(11) NOT NULL AUTO_INCREMENT,
   `status` int(11) DEFAULT '0',
   `hidden` int(11) DEFAULT '0',
   `checkinId` int(11) DEFAULT '-1',
   PRIMARY KEY (`id`),
   KEY `RoomLogIndex` (`roomId`),
   KEY `RoomLogIDIndex` (`id`),
   KEY `USERID` (`userID`)
 ) ENGINE=InnoDB AUTO_INCREMENT=1064216 DEFAULT CHARSET=utf8

I can also see that this query is running more number of times like 100000 times per day (nearly continuously).

SELECT count(*) from roomlogs where roomId=proomId and status='0';

Because of this query reads from the same table, does InnoDB block or create a lock on update query because I can see that when the above-stored procedure is running more number of times then this query is taking more time.

Here is the link for MySQL variables: https://docs.google.com/document/d/17_MVaU4yvpQfVDT83yhSjkLHsgYd-z2mg6X7GwvYZGE/edit?usp=sharing

Nilesh
  • 884
  • 3
  • 11
  • 22

1 Answers1

1

roomlogs needs this 'composite' index:

INDEX(userId, `status`, enterTime)

I added an index recently so previous rows are not indexed.

Not true. Adding an INDEX indexes the entire table.

The default index type is BTree; no need to explicitly specify it.

does the update query in the procedure make a lock?

It does some form of locking. What is the value of autocommit? Do you explicitly use BEGIN and COMMIT? Is the table ENGINE=InnoDB? Please provide SHOW CREATE TABLE.

MySQL.slow_logs table for each query the lock _time shows 0.

The INSERT you show seems to be inserting the same row as the UPDATE. Maybe you need INSERT ... ON DUPLICATE KEY UPDATE ...?

Don't "hide an index column in a function"; instead of DATEDIFF(roomlogs.enterTime,NOW()) = 0, do

AND enterTime >= CURDATE()
AND enterTime  < CURDATE() + INTERVAL 1 DAY

This allows the index to be used more fully.

KEY `RoomLogIndex` (`roomId`),  Change to  (roomId, status)
KEY `RoomLogIDIndex` (`id`),    Remove, redundant with the PK

Buffer pool in only 97,517,568 -- make it more like 9G.

Rick James
  • 135,179
  • 13
  • 127
  • 222
  • I have added show table information. also, it is inserting different rows as enter time will be different. – Nilesh Oct 05 '18 at 06:36
  • If two connections simultaneously run this stored procedure 'enter_room'. So, does MySQL create a lock on the rows and other queries has to wait? I have updated the code for the stored procedure for you to review. – Nilesh Oct 05 '18 at 22:00
  • @Nilesh - You should use `BEGIN` ... `COMMIT` around the SQL that should be treated in an atomic way. – Rick James Oct 05 '18 at 22:03
  • Thanks for your quick response. I do not want to block the queries. As I just want to know that If MySQL does do it by default? And it could be the reason that this stored procedure is taking more time than expected. – Nilesh Oct 05 '18 at 22:07
  • You probably _do_ want to block -- else strange anomalies can happen between the update and insert. Answer my other questions so we can get into details. – Rick James Oct 06 '18 at 03:26
  • Here are my indexs: PRIMARY KEY (`id`), KEY `RoomLogIndex` (`roomId`), KEY `RoomLogIDIndex` (`id`), KEY `USERID` (`userID`) – Nilesh Oct 07 '18 at 00:27
  • Add the composite index I suggested. – Rick James Oct 07 '18 at 01:25
  • What I have experienced is that when there are more queries running in the MySql then the time taken by this stored procedure is very high. While running on the fewer queries the time is good. – Nilesh Oct 08 '18 at 18:50
  • @Nilesh - Without a good index, those competing connections are stumbling over each other. – Rick James Oct 08 '18 at 19:56
  • I have added combined index as you suggested but it does not seem to helped much, Still query is taking more time than expected. – Nilesh Oct 09 '18 at 15:46
  • @Nilesh - Hmmm... Please provide `EXPLAIN SELECT ...` after adding the new index. – Rick James Oct 09 '18 at 18:23
  • I have edited USERID as combined index of (userId, `status`, enterTime) – Nilesh Oct 10 '18 at 06:20
  • Here is the output of: explain select * from roomlogs where userId = 'ncdwived' and roomlogs.`status` = 0 and DATEDIFF(roomlogs.enterTime,NOW()) = 0; id:1 select_type: SIMPLE table: roomlogs type: ref possible_keys: USERID key: USERID key_len: 58 ref: const,const rows: 7 Extra :Using index condition – Nilesh Oct 10 '18 at 06:21
  • What I have experienced is that when there are more queries running in the MySql then the time taken by this stored procedure is very high. While The other queries running at the same time are fine. that is why I doubt that this procedure creates a lock and block other query of same stored procedure? – Nilesh Oct 10 '18 at 06:27
  • See my added text about how to test `enterTime`. The "lock and block" is probably on the rows of the table more than on the Stored Procedure. Hence, I have focused on speeding up the query. – Rick James Oct 10 '18 at 16:14
  • Hello Rick, I have tried your suggestions but the time taken by query does not seem to improve much. Should I increase my max_connections OR the buffer size? currently, max_connections is 151. – Nilesh Oct 16 '18 at 20:30
  • @Nilesh - `max_connections` has no direct impact on query performance. What is the buffer_pool_size now? How much data do you have? What does `EXPLAIN SELECT ...` say. – Rick James Oct 16 '18 at 20:47
  • My InnoDB buffer pool size is 97517568. I am running a windows server of 16 GB RAM. The InnoDB buffer pool instance is 8. – Nilesh Oct 16 '18 at 23:34
  • Here is the output of: explain select * from roomlogs where userId = 'ncdwived' and roomlogs.status = 0 and entertime between curdate() and curdate() + interval 1 day; id:1 select_type: SIMPLE table: roomlogs type: range possible_keys: USERID key: USERID key_len: 64 ref: NULL ,const rows: 1 Extra :Using index condition – Nilesh Oct 16 '18 at 23:40
  • I added some stuff to address 2 of your comments. – Rick James Oct 17 '18 at 01:01
  • Thank you, Rick. You have helped so much. I will do mentioned changes. Also, do you have a reasoning for why the only enter_room is taking so much more time only when in the MySQL server has more queries while at the same time, other stored procedures(including the count query that I have mentioned) are taking a just little bit more than normal time? Doesn't MySQL create a lock on the rows or the table due to count queries? – Nilesh Oct 17 '18 at 05:41
  • @Nilesh - It could be locks, it could be caching. It could be something else. Increasing the buffer_pool_size is likely to eliminate caching as the problem. For other issues, see [_this_](http://mysql.rjweb.org/doc.php/mysql_analysis) for what we can do next. – Rick James Oct 17 '18 at 16:41
  • Create innodb_buffer_pool size to 2G. But buffer is only filling up around 9% all the time. Also, the time remains the same as of now. DO I need to configure some other variables? – Nilesh Oct 22 '18 at 18:23
  • @Nilesh - The buffer_pool is the main tunable. If it is not filling up, then, yeah, the "time remains the same". I'll study the rest of the variables/status in a few days. – Rick James Oct 23 '18 at 04:51
  • Here I have added my innodb status when more queries are running. If this can help. https://stackoverflow.com/questions/53001402/mysql-slow-performance-because-of-locking?noredirect=1#comment92907011_53001402 – Nilesh Oct 26 '18 at 04:42