0

I implemented a very simple Streaming Analytics query:

SELECT
    Collect()
FROM
    Input TIMESTAMP BY ts
GROUP BY
    TumblingWindow(second, 3)

I produce on an event hub input with a python script:

...
iso_ts = datetime.fromtimestamp(ts).isoformat()
data = dict(ts=iso_ts, value=value)
msg = json.dumps(data, encoding='utf-8')
# bus_service is a ServiceBusService instance
bus_service.send_event(HUB_NAME, msg)
...

I consume from a queue:

...
while True:
    msg = bus_service.receive_queue_message(Q_NAME, peek_lock=False)
    print msg.body
...

The problem is that I cannot see any error from any point in the Azure portal (the input and the output are tested and are ok), but I cannot get any output from my running process!

I share a picture of the diagnostic while the query is running: enter image description here

Can somebody give me an idea for where to start troubleshooting?

Thank you so much!

UPDATE

Ok, I guess I isolated the problem.
First of all, the query format should be like this:

SELECT
  Collect()
INTO
  [output-alias]
FROM
  [input-alias] TIMESTAMP BY ts
GROUP BY
  TumblingWindow(second, 3)

I tried to remove the TIMESTAMP BY clause and everything goes well; so, I guess that the problem is with that clause.

I paste an example of JSON-serialized input data:

{
  "ts": "1970-01-01 01:01:17",
  "value": "foo"
}

One could argue that the timestamp is too old (seventies), but I also tried with current timestamps and I didn't get any output and any error on the input.

Can somebody imagine what is going wrong? Thank you!

affo
  • 453
  • 3
  • 15
  • What does Azure portal's monitoring page say, do you see input events and were there any errors? Also, was there anything in the operation logs? – Vignesh Chandramohan Aug 01 '16 at 17:36
  • No, I can't see nothing! it is like there is no event at all. This is strange, because I fixed 2 bugs in body encoding and timestamp format and they were signalled. So I guess that the input was read by the topology... – affo Aug 01 '16 at 18:40
  • Do you know if in `FROM Input TIMESTAMP BY ts` the string `Input` should match some user defined name? For example, the input alias? – affo Aug 01 '16 at 18:43
  • The string Input should be the name of the input. When you say " it is like there is no event at all". Do you mean input or output or both? What about input errors? – Vignesh Chandramohan Aug 01 '16 at 18:48
  • My input alias was `Input` and now I turned it to `input` with a lower case "i". I don't know why, but it seems that something has changed... Now I activated the monitoring stuff and I am updating the question. – affo Aug 01 '16 at 19:26
  • Vignesh what do you mean with "the name of the input"? Do you mean input alias? – affo Aug 01 '16 at 19:30

2 Answers2

1

I discovered that my question was a duplicate of Basic query with TIMESTAMP by not producing output.

So, the solution is that you cannot use data from the seventies, because streaming analytics will consider that all the tuples are late and will drop them.

I re-tried to produce in-time tuples and, after a long latency, I could see the output.

Thanks to everybody!

Community
  • 1
  • 1
affo
  • 453
  • 3
  • 15
0

Can you check the Service Bus queue from Azure portal for number of messages received?

chetangm
  • 101
  • 3