2

we found a new service that could be quite useful to monitor what is happening on our servers and in our applications. Problem is that it is SaaS, so we can't just install the tool on our servers, but have to send the data over the wire to the servers of the company. Additionally, the agent and daemon collecting and sending the data over aren't open source, so I can't just look at the code.

Fortunately, they offer quite a configuration option to route all the traffic through a proxy. My idea is to install a simple proxy that logs the traffic in an easily readable format.

What's the most simple proxy that can log traffic to a file in a handy format?

Thanks.

janpio
  • 155
  • 2
  • 6

2 Answers2

2

If you are specifically looking for proxies then squid is your answer.

If you are just looking for the end result you can use a combination of tools to gather data. (for example: you dont want all traffic sent through a single server, maybe in a multi site network. All data gets logged in a central location for reporting but all users dont have to connect to a single point for access.)

just to name a few useful tools:
vnstat - Monitors data usage
tcpdump - gives you all network data - filter out what you dont want
bandwidthd - shows you potential bandwidth at any given time (can graph over time)
wireshark - log data at the packet level (good for capturing login id's and context of the sites visited(IE searching for keywords in page content)) filter what you want to log
cacti - interface monitoring gui

if you needed more detailed statistical info you could use nagios or zenoss which both have great reporting tools for just about anything you need.

DaffyDuc
  • 512
  • 2
  • 7
  • A proxy was the only hammer I knew, but of course I was looking for the end result. Thanks for assuming that. Tcpdump even seems better than a proxy, othrwise the service could just send some traffic not using the configured proxy. – janpio Mar 04 '12 at 10:28
  • 1
    Be aware that you want to filter it out before you log it. TCPdump will log everything and blow up your drive space if you leave it unmonitored. its amazing how much data that app can generate. – DaffyDuc Mar 05 '12 at 14:33
1

HTTP Proxy: Squid.

Lower level than that, perhaps you could just use netcat (nc) and tcpdump in the middle. You might be able to do it with IPTables, come to think of it.

Tom O'Connor
  • 27,480
  • 10
  • 73
  • 148