-4

Are there any technologies that can take raw semi-structured, schema-less big data input (say from HDFS or S3), perform near-real-time computation on it, and generate output that can be queried or plugged in to BI tools?

If not, is anyone at least working on it for release in the next year or two?

tshepang
  • 12,111
  • 21
  • 91
  • 136
jbx72487
  • 91
  • 6
  • 2
    No, but if you come up with a solution to this fundamental computing problem, let us all know! ;-) :-) –  Jul 01 '12 at 12:27
  • This question needs to be more specific if you want an answer. What kind of computation? What volume of data? What BI tool do you want to receive the output? – Craig Citro Jul 09 '12 at 23:04

2 Answers2

4

enter image description here

There are some solutions with big semistructured input and queried output, but they are usually

  1. unique

  2. expensive

  3. secret enough

If you are able to avoid direct computations using neural networks or expert systems, you will be close enough to low latency system. All you need is a team of brilliant mathematicians to make a model of your problem, a team of programmers to realize it in code and some cash to buy servers and get needed input/output channels for them.

tshepang
  • 12,111
  • 21
  • 91
  • 136
Pavel Oganesyan
  • 6,774
  • 4
  • 46
  • 84
0

Have you taken a look at Splunk? We use it to analyze Windows Event Logs and Splunk does an excellent job indexing this information to allow for fast querying of any string that appears in the data.

HeatfanJohn
  • 7,143
  • 2
  • 35
  • 41