8

"Big Data" is a big trending term on the internet (especially on the web sites of big companies selling data management services). For example, I came across this article while reading the Irish Times in an airport. It contained this recent claim from IBM which has been frequently repeated elsewhere (my emphasis):

Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone.

Is this just marketing hype or is the claim credible?

NB it might also be worth asking whether the claim holds up with something other than a vague definition of "data".

Sklivvz
  • 78,578
  • 29
  • 321
  • 428
matt_black
  • 56,186
  • 16
  • 175
  • 373
  • This really hinges on how you define "data", but since it's from IBM they probably mean "retrievable, usable information recorded on a persistent medium". – Tacroy Aug 10 '12 at 17:02
  • I’m unsure how evidence for this would look like but as a partial answer, this is [more or less true in bioinformatics](http://www.ncbi.nlm.nih.gov/genbank/genbankstats-2008/) (note that the data is outdated, growth has accelerated since then) which is one of the bigger data producer recently. It’s almost certainly true in physics (think LHC), too. But I’m suspecting that the most data easily comes from other sources, namely people creating content. – Konrad Rudolph Aug 10 '12 at 19:37
  • Grammar pedants might want to tell me whether the title should start "**Have** 90% of the world's data..." – matt_black Aug 10 '12 at 21:51
  • @matt_black Per wiktionary, on 'data': 'This word is more often used as an uncountable noun with a singular verb than as a plural noun, with singular datum.' That being the case, and in the absence of anything indicating that the question refers specifically to a set of individual facts, 'has' seems correct to me. – MrTheWalrus Aug 12 '12 at 23:07
  • Video at 1080p60 are tremendous amount of data. 15-24Mbps is typical bitrate of most camcorders. Most smartphones can't do 1080p60 videos, but they usually do 1080p30 or 720p60. Producing videos at 100MB per minute of recording. For comparison, few years ago most smartphones could only record QVGA or VGA videos at 30fps or sometimes as low as 15fps. Or even not at all (eg. iPhone 3G). – vartec Aug 12 '12 at 23:18
  • The "recent claim from IBM" is dated back in 2013. Originally, the "last two years" would mean 2011-2012. But in 2015 today, "last two years" could mean 2013-2014. So any given answer to this question would be outdated every two years... How would one answer such tricky question? –  Nov 12 '15 at 14:46

1 Answers1

6

I have no rigorous research sources to cite, but based on reports like this I would judge the basic plausibility of this claim to be decent if the following hold up to scrutiny:

  1. Devices with cameras have proliferated significantly in the last 4-6 years with the advent of so-called 'Smartphones'.
  2. Those cameras have increasingly large megapixel counts in each generation.
  3. Video is also more and more frequently a feature of these devices.
  4. Cloud services and online social sites allow all this data to be retained (and counted) more reliably.
  5. HD video has also proliferated online on many popular video sites - movies and even HD streaming (which is recorded) of video games (eSports) and other events.

I would argue that 1-4 have a kind of multiplicative effect on each other... tens of millions more devices (2 years ago, Apple and Google activated 9 million phones a month) taking 8MPix pictures and HD videos would add up very quickly.

ZeroBugBounce
  • 161
  • 2