Counting substrings in a string

Question

How can I count the numbers of times a substring shows up in a string?

In this case, I'm looking for every time "connect.facebook.net/en_US/all.js" shows up in the HTML bodies of the top 300K internet sites (stored in httparchive).

Answer 1

You could use SPLIT() on the string, and count the number of records produced:

SELECT fb_times, COUNT(*) n_pages
FROM
  (SELECT COUNT(splits)-1 WITHIN RECORD AS fb_times
   FROM
     (SELECT SPLIT(body, 'connect.facebook.net/en_US/all.js') splits
      FROM [httparchive:runs.2014_08_15_requests_body]
      WHERE body CONTAINS 'connect.facebook.net/en_US/all.js'
        AND mimeType="text/html"
        AND page=url))
GROUP BY 1
ORDER BY 1

Note the usage of WITHIN RECORD to count how many sub-records SPLIT() produced.

Results:

Fb_times    N_pages
1           12,471
2           1,222
3           163
4           34
5           18
6           12
7           12
8           6
...         ...