1

Currently I'm using basic API gets to pull down logs from a cloud security vendor but it's not a very clean process. I would like to transition over to the event streaming (AMQPS) service they provide.

I've already setup the queue and have the relevant information on the cloud side but the issue I'm running into is figuring out the best way to accept that stream and dump them to syslog. I'm utilizing syslog-ng right now but it looks like it can only send to RabbitMQ and not accept it.

I'm currently looking at Pika to see if that would work to accept AMQPS and then funnel it to a log file but that's where I'm running into some problems. Any help would be greatly appreciated.

Thanks

Eric
  • 1,383
  • 3
  • 17
  • 34
  • What problems? Pika is a Python library implementing AMQP so it can both produce content to queues and consume them so you can imagine any kind of interface on top of it. – Patrick Mevzek Mar 20 '18 at 05:45
  • 1
    Hi, it's possible to add sources to syslog-ng using Python, you can contact the developers who can help you to get started (for example, open a github issue at https://github.com/balabit/syslog-ng/issues or use https://gitter.im/balabit/syslog-ng ). – Robert Fekete Mar 20 '18 at 07:28
  • [tag:graylog] can accept AMQPS. – Martin Schröder Mar 20 '18 at 09:31

1 Answers1

0

I found a github page made by walbit (https://github.com/walbit/AMP_API), that appears to be working correctly for me. I'm using the "consume_event_stream_by_name.py" and it dumps all of the input to standard out. I made a slight modification to send it to syslog as well and now syslog-ng is accepting it.

My only issue at the moment with this code is that it seems to be timing out from time to time or dying overall if connections aren't coming in. So I need to figure out what is going on with that.

#!/usr/bin/env python

import argparse
import pika
import pprint
import requests
import sys
import logging
import logging.handlers

# YOU NEED TO CREATE AN auth.py FILE WITH CLIENT_ID AND API_KEY STRINGS
from auth import CLIENT_ID, API_KEY

parser = argparse.ArgumentParser()
parser.add_argument('event_stream_name', metavar='event_stream_name',
                    nargs=1, help='event stream name')
parser.parse_args()
event_stream_name = parser.parse_args().event_stream_name[0]

api_endpoint = 'https://api.amp.cisco.com/v1/event_streams'

session = requests.Session()
session.auth = (CLIENT_ID, API_KEY)

event_streams = session.get(api_endpoint).json()['data']

event_stream = {}

for e in event_streams:
    if e['name'] is event_stream_name:
        event_stream = e

amqp_url = 'amqps://{user_name}:{password}@{host}:{port}'.format(
    **e['amqp_credentials'])
queue = e['amqp_credentials']['queue_name']
parameters = pika.URLParameters(amqp_url)
connection = pika.BlockingConnection(parameters)
channel = connection.channel()


parameters = pika.URLParameters(amqp_url)
connection = pika.BlockingConnection(parameters)
channel = connection.channel()

my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.INFO)
handler = logging.handlers.SysLogHandler(address = '/dev/log')
my_logger.addHandler(handler)

def callback(ch, method, properties, body):
#    print(" [x] Received meth:\t%r" % method)
#    print(" [x] Received prop:\t%r" % properties)
#    print(" [x] Received body:\t%r" % body)
#    print(body)
    my_logger.info('Fireamp: ' + body)


channel.basic_consume(callback, queue, no_ack=True)

print(" [*] Connecting to:\t%r" % amqp_url)
print(" [*] Waiting for messages. To exit press CTRL+C")
channel.start_consuming()
Eric
  • 1,383
  • 3
  • 17
  • 34