13

I have a bunch of linux virtual servers left over from a previous IT department. They have names like 'magic' or 'stuff'. I'm not quite sure what they're doing...or if I need them...

How would you guys and gals go about finding out the purpose of these machines? (besides turning them off and seeing what breaks)

blsub6
  • 1,131
  • 6
  • 25
  • 45

8 Answers8

20

A couple places to start:

  • listening services (netstat) - this should, generally speaking, give you a decent idea of what's going on with the system.
  • /root/.bash_history (or that of other users, if they didn't use root) - whatever's been going on on the console will, ideally, be related to the purpose of the system.
  • /var/log - take a glance at the standard logs, and look for anything application related.
  • Installed packages - this is specific to the distribution of linux that they're running, but if the logs are there, take a look. /var/log/dpkg.log, /var/log/yum.log, etc.
Shane Madden
  • 114,520
  • 13
  • 181
  • 251
12

Hardly scientific I know but if you get permission from your management I'd consider pausing the VMs - you'll find out if they're important quicker that you'd think, if it stays paused with nobody complaining...well that tells you something else.

Seriously though you could spend a career trying to figure them out without every truly knowing everything they do. Pausing them may seem odd/draconian but in the absence of documentation I'm sure you could sell the idea to management, as a one-off at first to see how it goes anyway.

Chopper3
  • 101,299
  • 9
  • 108
  • 239
  • 4
    +1 - The only way to find out what services something is running is to turn it off. My last job had a Windows NT4 "print server" that was up years after it should have died. The day it was turned off a whole bunch of stuff broke that nobody knew was running on that box. – voretaq7 Mar 07 '11 at 22:06
  • 1
    Pause them and wait. And after two months have passed and some random user complains that a share, a shortcut or something else is not working (but it worked!) turn them on. – adamo Mar 07 '11 at 22:59
  • 5
    @adamo...momentarily to migrate the data/functionality to a recognised supported machine/VM then switch if off again. – Chopper3 Mar 07 '11 at 23:40
7

I was surprised to see that the first answer suggested wasn't ps -ef, so I'll add it: if you want to know what a system is doing right now, read the process list, paying particular attention to what root is up to, and whether there are processes owned by conspicuously-named users (mysql, named, etc).

I'd then compare my process list against lsof run as root to see which processes are listening on the network, and which are holding open files. Typically this gives you a pretty good picture of the long-running processes on the box, which are commonly its main function.

Notable exceptions include mail - see local syslog and mailq for details on what's being processed by sendmail - and inted-type run-on-demand services, for which /etc/xinetd.conf is a good bet, at least for most recent Redhat-based Linuxes.

Hope that helps; let us know if you run into something in particular we can help identify!

Jeff Albert
  • 1,987
  • 9
  • 14
1

I would start by seeing what services are running... Then attempt to match those to what they are hosting. Do NOT under any circumstance power off what you have no idea it's doing as you could break what ever it is running an if its mission critical( if that is the route your dying to take, pause them)... You should also check to see if there any sort of documentation.

Jacob
  • 9,204
  • 4
  • 45
  • 56
1

Oh dear, that's a fun one.

Do you have any idea what they're used for? Can you narrow it down to "these were used for network services", or could it really be anything at all?

I would say a packet capture on each server is necessary, along with an audit of all running services. Locate the config files for each running service and check when the files were last updated - that will give you a clue as to whether something has been customized, and if so, how long ago.

You can also run a port scan on each server to see which ports are open and responding.

You can get clues by querying known network services - EG, DNS, LDAP, etc. You should be able to find a list of all DNS servers for a particular zone by digging for NS records. Bear in mind that you may end up with a longer list of NS records than there are actually active DNS servers, but it'll give you a starting point.

None of these methods are sure fire by themselves, but if you throw multiple audit methods at a particular box your chances of finding everything worth finding is enhanced.

Good luck!

Jeremy
  • 938
  • 2
  • 7
  • 18
0

A port scan would reveal any network accessible services

From the server locally: nmap 127.0.0.1

Or you can tell nmap to scan a certain subnet/mask

sreimer
  • 2,218
  • 15
  • 17
0

One other angle is look at what is configured to connect to the servers. If foozle.example.com is configured in the CEO's email client, it is probably the mail server. FTP clients probably point towards a web server of some sort. Etc, etc.

Wyatt Barnett
  • 725
  • 5
  • 14
  • While that would work the problem is that every other machine and possibly even every user account on those machines would have to be checked, rather than just the target machines. – John Gardeniers Mar 08 '11 at 07:11
  • Not really -- if these are in fact servers, checking a sample should tell you what the bulk of the boxes are. Or at least the commonly internally accessed. It strikes me that you could also look at firewall rules to cover the externally accessible services. – Wyatt Barnett Mar 08 '11 at 12:54
0

ps -ef for processes, netstat -a for services listening and tcpdump to see what traffic is going back and forth are great suggestions. In addition, since it's Linux, there's a good chance there's a firewall running - check out the rules set-up for it, should give you a good clue what services are expected to be used on this host and remote hosts that this host connects to. e.g. iptables --list Of course, what firewall is there is another thing to be checked out, try lsmod to look for firewall modules and check out /var/log

Bob T
  • 1