1

I have a simple query. All I am trying to do is combine geolocation data into a primary table so I dont need to use a join when pulling data. Yeah a join is fine but I want all the data in a single table...

Here is my query:

> explain UPDATE test AS l 
SET l.country_name1 = ( 
select country_name
from ip2location_db9 
    where  
        l.ip >= ip_from
        ORDER BY ip_from DESC
        LIMIT 1) 

******************** 1. row *********************
           id: 1
  select_type: UPDATE
        table: l
   partitions: 
         type: ALL
possible_keys: 
          key: 
      key_len: 
          ref: 
         rows: 10
     filtered: 100.00
        Extra: 
******************** 2. row *********************
           id: 2
  select_type: DEPENDENT SUBQUERY
        table: ip2location_db9
   partitions: 
         type: index
possible_keys: idx_ip_from,idx_ip_from_to,ip_from_to,ip_from
          key: idx_ip_from
      key_len: 5
          ref: 
         rows: 1
     filtered: 33.33
        Extra: Using where; Backward index scan
2 rows in set

Simple right? Except that this query for 10 lines takes 130+ seconds?.. The select statement when run by itself is super fast (0.00 sec) and works perfectly.

explain select country_name
from ip2location_db9
    where  
        ip_from <= INET_ATON('114.160.63.108')
        ORDER BY ip_from DESC
       LIMIT 1

******************** 1. row *********************
           id: 1
  select_type: SIMPLE
        table: ip2location_db9
   partitions: 
         type: range
possible_keys: idx_ip_from,idx_ip_from_to
          key: idx_ip_from
      key_len: 5
          ref: 
         rows: 1949595
     filtered: 100.00
        Extra: Using index condition
1 rows in set

However when I use it with the update statement with only 14 rows in the test table, the query takes a painful 130 seconds.

Tables are set up as follows:

CREATE TABLE `test` (
  `ip` int(10) unsigned DEFAULT NULL,
  `country_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `region_name` varchar(128) COLLATE utf8_bin DEFAULT NULL,
  `city_name` varchar(128) COLLATE utf8_bin DEFAULT NULL,
  `latitude` double DEFAULT NULL,
  `longitude` double DEFAULT NULL,
  `zip_code` varchar(30) COLLATE utf8_bin DEFAULT NULL,
  `data` varchar(512) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

CREATE TABLE `ip2location_db9`(
    `ip_from` INT(10) UNSIGNED,
    `ip_to` INT(10) UNSIGNED,
    `country_code` CHAR(2),
    `country_name` VARCHAR(64),
    `region_name` VARCHAR(128),
    `city_name` VARCHAR(128),
    `latitude` DOUBLE,
    `longitude` DOUBLE,
    `zip_code` VARCHAR(30),
    INDEX `idx_ip_from` (`ip_from`),
    INDEX `idx_ip_to` (`ip_to`),
    INDEX `idx_ip_from_to` (`ip_from`, `ip_to`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

Note that when running the update query, the hard drive activity goes crazy high utilization. See? Nothing fancy. Table data matches. Rows are properly indexed.

Explain shows that is uses the index just as it does when I perform the select by itself:

******************** 1. row *********************
           id: 1
  select_type: UPDATE
        table: l
   partitions: 
         type: ALL
possible_keys: 
          key: 
      key_len: 
          ref: 
         rows: 14
     filtered: 100.00
        Extra: 
******************** 2. row *********************
           id: 2
  select_type: DEPENDENT SUBQUERY
        table: ip2location_db93
   partitions: 
         type: index
possible_keys: idx_ip_from,idx_ip_from_to
          key: idx_ip_from
      key_len: 5
          ref: 
         rows: 1
     filtered: 33.33
        Extra: Using where
2 rows in set

So what could I possibly been doing wrong that would cause the query to take 2 min to run for 14 lines? Its not a hardware thing. i7 6990, 32gb ram, running off an ssd. Not the fastest in the world but I can manually update 14 rows faster than this query can...

I have spent an excess of time searching trying to find out why this takes so long. I assume that I am just not searching correctly. Perhaps I dont know a specific term or something that would point me in the right direction. I am not a DB guy. Just doing this for a work thing.

Hoping you guys can save my sanity..

Adding more info...

I have tried to make this query work many many ways. Other questions on stack said to avoid the subquery and use a join. Ok that was the first thing I tried but I cant get the query to use the indexes I built.

> explain UPDATE test AS l 
    JOIN 
        ip2location_db9 AS h
    ON  
        l.ip <= h.ip_from and l.ip >= h.ip_to
    SET 
        l.country_name1 = h.country_name



******************** 1. row *********************
           id: 1
  select_type: UPDATE
        table: l
   partitions: 
         type: ALL
possible_keys: ip
          key: 
      key_len: 
          ref: 
         rows: 10
     filtered: 100.00
        Extra: 
******************** 2. row *********************
           id: 1
  select_type: SIMPLE
        table: h
   partitions: 
         type: ALL
possible_keys: idx_ip_from,idx_ip_to,idx_ip_from_to,ip_from_to,ip_from,ip_to
          key: 
      key_len: 
          ref: 
         rows: 3495740
     filtered: 11.11
        Extra: Range checked for each record (index map: 0x3F)
2 rows in set

Does an update with a join get any easier than that?
Even using force index doesn't get the query to use an index.

Ive tried this query so many ways

UPDATE test
JOIN (
   SELECT * FROM ip2location_db9 
) AS t1 
ON (test.ip <= t1.ip_from and test.ip >= t1.ip_to)
SET test.country_name1 = t1.country_name


UPDATE test l,
    (SELECT 
        *
    FROM
        ip2location_db9,test
    WHERE
        test.ip >= ip_from 
    ORDER BY ip_from DESC
    LIMIT 1) AS PreQualified 
SET 
    l.country_name1 = PreQualified.country_name

Nothing works! What am I doing wrong?

  • Looks Like Duplicate: [why-is-this-mysql-query-extremely-slow](https://stackoverflow.com/questions/17590666/why-is-this-mysql-query-extremely-slow) – always-a-learner Jun 20 '17 at 04:09
  • 1
    You Should know how to ask a good question. [how-to-ask](https://stackoverflow.com/help/how-to-ask) – always-a-learner Jun 20 '17 at 04:11
  • can you provide some sample data and the result you're looking for? Updates like this generally perform better using joins in the `UPDATE` – But those new buttons though.. Jun 20 '17 at 04:38
  • 2
    @ankitsuthar This has nothing to do with innodb vs myisam nor about the select statement. AS I said, the select works perfectly. Its when I use it in an UPDATE that it goes wrong. I think, with exception to the title, the contents fits your 'how to ask' answer. – user2737277 Jun 20 '17 at 05:44
  • @user2737277 - some real sample data would be useful. I created those tables as you mentioned and populated them with about 20 rows each. First problem is there is no `country_name1` column in your table def so not sure if that's a typo or if you've actually got things set up differently. When I did fix the query to use existing column name it ran successfully in 41ms. Something is either wrong with your server or you're dealing with something much different than what you posted here. – But those new buttons though.. Jun 23 '17 at 04:08
  • How many rows in each table? What version of MySQL? Are you getting the 'right' answer? Or should it be ORDER BY .. ASC? – Rick James Jun 30 '17 at 22:20

0 Answers0