Cygnus does not recognize my service-path

Question

I have moved my Fiware structure to the Amazon EC2 service, the configuration of my agent is as follows:

cygnus-ngsi.sources = http-source
cygnus-ngsi.sinks = mongo-sink
cygnus-ngsi.channels = mongo-channel

cygnus-ngsi.sources.http-source.channels = mongo-channel
cygnus-ngsi.sources.http-source.type = org.apache.flume.source.http.HTTPSource
cygnus-ngsi.sources.http-source.port = 5050
cygnus-ngsi.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.NGSIRestHandler
cygnus-ngsi.sources.http-source.handler.notification_target = /notify
cygnus-ngsi.sources.http-source.handler.default_service = default
cygnus-ngsi.sources.http-source.handler.default_service_path = /sevilla
cygnus-ngsi.sources.http-source.handler.events_ttl = 2
cygnus-ngsi.sources.http-source.interceptors = ts
cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp

cygnus-ngsi.sinks.mongo-sink.type = com.telefonica.iot.cygnus.sinks.NGSIMongoSink
cygnus-ngsi.sinks.mongo-sink.channel = mongo-channel
cygnus-ngsi.sinks.mongo-sink.enable_encoding = true
cygnus-ngsi.sinks.mongo-sink.enable_grouping = false
cygnus-ngsi.sinks.mongo-sink.enable_name_mappings = false
cygnus-ngsi.sinks.mongo-sink.enable_lowercase = false
cygnus-ngsi.sinks.mongo-sink.data_model = dm-by-service-path
cygnus-ngsi.sinks.mongo-sink.attr_persistence = column
cygnus-ngsi.sinks.mongo-sink.mongo_hosts = ******com:15959
cygnus-ngsi.sinks.mongo-sink.mongo_username = ********
cygnus-ngsi.sinks.mongo-sink.mongo_password = ********
cygnus-ngsi.sinks.mongo-sink.db_prefix = sth_
cygnus-ngsi.sinks.mongo-sink.collection_prefix = sth_
cygnus-ngsi.sinks.mongo-sink.batch_size = 1
cygnus-ngsi.sinks.mongo-sink.batch_timeout = 120
cygnus-ngsi.sinks.mongo-sink.batch_ttl = 10
cygnus-ngsi.sinks.mongo-sink.data_expiration = 0
cygnus-ngsi.sinks.mongo-sink.collections_size = 0
cygnus-ngsi.sinks.mongo-sink.max_documents = 0
cygnus-ngsi.sinks.mongo-sink.ignore_white_spaces = true

cygnus-ngsi.channels.mongo-channel.type = com.telefonica.iot.cygnus.channels.CygnusMemoryChannel
cygnus-ngsi.channels.mongo-channel.capacity = 15000
cygnus-ngsi.channels.mongo-channel.transactionCapacity = 11000

The problem is that when you send the data to MongoDB, it does not send them to the service-path that is configured in the agent, obtaining this response in cygnus.log

Database: sth_default, Collection: sth_x002f srv = default | subsrv = /

What happens so that it does not recognize my service-path?

edit: I put my cygnus.log

time=2018-05-30T09:34:41.359Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.359Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.361Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.361Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.361Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.361Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.361Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.362Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.362Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[930] : Added sinks: mongo-sink Agent: cygnus-ngsi
time=2018-05-30T09:34:41.362Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.362Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2018-05-30T09:34:41.372Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=validateConfiguration | msg=org.apache.flume.conf.FlumeConfiguration[140] : Post-validation flume configuration contains configuration for agents: [cygnus-ngsi]
time=2018-05-30T09:34:41.372Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=loadChannels | msg=org.apache.flume.node.AbstractConfigurationProvider[150] : Creating channels
time=2018-05-30T09:34:41.380Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=create | msg=org.apache.flume.channel.DefaultChannelFactory[40] : Creating instance of channel mongo-channel type com.telefonica.iot.cygnus.channels.CygnusMemoryChannel
time=2018-05-30T09:34:41.384Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=loadChannels | msg=org.apache.flume.node.AbstractConfigurationProvider[205] : Created channel mongo-channel
time=2018-05-30T09:34:41.385Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=create | msg=org.apache.flume.source.DefaultSourceFactory[39] : Creating instance of source http-source, type org.apache.flume.source.http.HTTPSource
time=2018-05-30T09:34:41.403Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=configure | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[156] : [NGSIRestHandler] Startup completed
time=2018-05-30T09:34:41.413Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=create | msg=org.apache.flume.sink.DefaultSinkFactory[40] : Creating instance of sink: mongo-sink, type: com.telefonica.iot.cygnus.sinks.NGSIMongoSink
time=2018-05-30T09:34:41.423Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=getConfiguration | msg=org.apache.flume.node.AbstractConfigurationProvider[119] : Channel mongo-channel connected to [http-source, mongo-sink]
time=2018-05-30T09:34:41.427Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=startAllComponents | msg=org.apache.flume.node.Application[138] : Starting new configuration:{ sourceRunners:{http-source=EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }} sinkRunners:{mongo-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@46157e43 counterGroup:{ name:null counters:{} } }} channels:{mongo-channel=com.telefonica.iot.cygnus.channels.CygnusMemoryChannel{name: mongo-channel}} }
time=2018-05-30T09:34:41.427Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=startAllComponents | msg=org.apache.flume.node.Application[145] : Starting Channel mongo-channel
time=2018-05-30T09:34:41.468Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=register | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[110] : Monitoried counter group for type: CHANNEL, name: mongo-channel, registered successfully.
time=2018-05-30T09:34:41.469Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[94] : Component type: CHANNEL, name: mongo-channel started
time=2018-05-30T09:34:41.469Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=startAllComponents | msg=org.apache.flume.node.Application[173] : Starting Sink mongo-sink
time=2018-05-30T09:34:41.469Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=startAllComponents | msg=org.apache.flume.node.Application[184] : Starting Source http-source
time=2018-05-30T09:34:41.472Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[354] : [mongo-sink] Startup completed
time=2018-05-30T09:34:41.548Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=register | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[110] : Monitoried counter group for type: SOURCE, name: http-source, registered successfully.
time=2018-05-30T09:34:41.548Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[94] : Component type: SOURCE, name: http-source started
time=2018-05-30T09:34:42.348Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=main | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication[301] : Starting a Jetty server listening on 0.0.0.0:8081 (Management Interface)
time=2018-05-30T09:35:03.570Z | lvl=INFO | corr=bafb7ba2-63ec-11e8-8f79-0ac12ccf4042 | trans=9ce00f30-1053-44f8-ae38-ef27a7933b09 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[286] : [NGSIRestHandler] Starting internal transaction (9ce00f30-1053-44f8-ae38-ef27a7933b09)
time=2018-05-30T09:35:03.571Z | lvl=INFO | corr=bafb7ba2-63ec-11e8-8f79-0ac12ccf4042 | trans=9ce00f30-1053-44f8-ae38-ef27a7933b09 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[304] : [NGSIRestHandler] Received data ({"subscriptionId":"5b0e7024cbff54ce8948ebfa","originator":"localhost","contextResponses":[{"contextElement":{"type":"ExerciseAction","isPattern":"false","id":"7120","attributes":[{"name":"location","type":"geo:point","value":"38.3763726, -32.1864475","metadatas":[{"name":"crs","type":"Text","value":"WGS84"}]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]})
time=2018-05-30T09:35:03.631Z | lvl=INFO | corr=bafb7ba2-63ec-11e8-8f79-0ac12ccf4042 | trans=9ce00f30-1053-44f8-ae38-ef27a7933b09 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=persistAggregation | msg=com.telefonica.iot.cygnus.sinks.NGSIMongoSink[361] : [mongo-sink] Persisting data at NGSIMongoSink. Database: sth_default, Collection: sth_x002f, Data: [Document{{recvTime=Wed May 30 09:35:03 UTC 2018, entityId=7120, entityType=ExerciseAction, location=38.3763726, -32.1864475}}]
time=2018-05-30T09:35:04.394Z | lvl=INFO | corr=bafb7ba2-63ec-11e8-8f79-0ac12ccf4042 | trans=9ce00f30-1053-44f8-ae38-ef27a7933b09 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[553] : Finishing internal transaction (bafb7ba2-63ec-11e8-8f79-0ac12ccf4042)

edit2: cygnus subscription configuration

<?php
$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, "http://publicip:1026/v2/subscriptions");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, FALSE);

curl_setopt($ch, CURLOPT_POST, TRUE);

curl_setopt($ch, CURLOPT_POSTFIELDS, "{
  \"description\": \"One subscription to rule them all\",
  \"subject\": {
    \"entities\": [
      {
        \"idPattern\": \".*\",
        \"type\": \"ExerciseAction\"
      }
    ]
  },
  \"notification\": {
    \"http\": {
      \"url\": \"http://localhost:5050/notify\"
    },
    \"attrs\": [
      \"id\",
      \"startTime\",
      \"name\",
      \"agent\",
      \"exerciseCourse\",
      \"location\",
      \"properties\",
      \"distance\",
      \"exerciseType\",
      \"endTime\",
      \"sportsTeam\"
    ],\"attrsFormat\":\"legacy\"
  },
  \"expires\": \"2020-04-05T14:00:00.00Z\"
}");

curl_setopt($ch, CURLOPT_HTTPHEADER, array(
  "Content-Type: application/json"));

$response = curl_exec($ch);
curl_close($ch);

var_dump($response);

?>

SOLUTION: Simply add the headers in this way for both subscriptions and to create an entity

$headers = array(
   'Content-Type: application/json','Fiware-Service: default','Fiware-ServicePath: /mypath'
);

curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

anmunoz · Answer 1 · 2018-06-06T13:10:39.177

1

For avoiding the override of the default values for Fiware-Service-Path and Fiware-Service configured in Cygnus, You have to include the Headers Fiware-Service-Path and Fiware-Service with their corresponding values in your subscription request.

edited Jun 06 '18 at 13:10

answered May 29 '18 at 16:38

anmunoz

131
6

Thank you anmunoz for your response! But it's not posible, in my other fiware achitecture on FiwareLab I used the same agent, and there was no problem, my agent create the response on the service path that I indicated. – franciclogreen May 30 '18 at 07:48
If you haven't had this behavior before the problem maybe is not directly related to Cygnus. Please, can you update your question with some additional information, like cygnus logs and the notification sent by Orion? – anmunoz May 30 '18 at 09:14
According to your logs, Cygnus is not receiving the service and the service path, in other words, the Orion notification doesn't include that Headers or include some values that are overriding the default values configured in Cygnus. Please, check how you made your subscription to Orion and if you include the headers in the subscription. Also, you can add your subscription request for checking it. – anmunoz May 30 '18 at 12:52
Thank you anmunoz for your response! I edit my post and I attach my subscription to cygnus. Is the same configuration that I put on my fiware lab structure, and it worked well. – franciclogreen May 30 '18 at 23:44
Hi Fran, I notice in your subscription code that you set the Header to false and after that, you send a new Header with the Content-type information `curl_setopt($ch, CURLOPT_HTTPHEADER, array( "Content-Type: application/json"));` , so you have to do the same thing but including the Headers for Fiware Service and Fiware Service Path ` Fiware-Service: myservice Fiware-ServicePath: /myservicepath ` – anmunoz May 31 '18 at 13:07
Thank you anmunoz I have solved my problem, I post the solution. Best regards!! – franciclogreen Jun 05 '18 at 08:11
Thanks for the feedback franciclogreen, with your update I am going to edit my answer. – anmunoz Jun 06 '18 at 13:04

Cygnus does not recognize my service-path

1 Answers1