Ive always wondered, what can we do if a virus or worm happens to affect the main file server?
What if from the file server the clients get infected by said worm/virus?
How would you clean it up? Where would you even start in a compromised environment that has anti-virus in place?
"infects everything" as in files on a server, or everything as in 800 workstations are slamming the network with attempts to infect each other?
The "right" answer would be wipe and reinstall from backups. The practical answer isn't always so straightforward.
Most viruses now tend to be straightforward in that they're not infecting files, but infecting a few key files or acting as droppers, so your files aren't usually going to be spreading the malware. If you're hit by most popular self-propagating malware today there's usually a targeted disinfector available from popular AV sites. The hard part (usually) is getting to the site, as many of these malware programs will attempt to mask themselves and try disabling AV programs, DNS requests to AV sites, etc. so you'll end up looking for a way to get into the site to get the tool in the first place.
We've had large scale infections of our systems with a worm. The key for us was mitigation of risk. Out of 800+ systems, only a small fraction of them aren't running Deep Freeze, a program that restores computers to their original state when restarted. So for those systems we can use the "star trek" method of fixing the computers in the network. Shut down everything. All at once.
That left us with administrative systems, certain staff, and servers to repair. Many of them were already immune due to keeping up with patches. The others had a targeted disinfector run, then re-checked with a couple AV programs to verify they weren't showing signs of infection.
We also used tools to scan the network for systems that weren't patched or had remote signs of the infection (it was a worm that had a network signature with the right scan method) so we could target our efforts on what to triage for repairs. After all the signs of infection were off the network, we restarted all the Deep Freeze systems.
(secondary note - we also have outgoing port 25 blocked for all but our mail server to prevent getting our domain blacklisted)
So the best way to prevent that issue is to work on mitigation of risk, in our view. Students don't have profiles; makes it harder to spread downloaded (or drive-by) malware. Permissions segregate data in the home directories of servers. Deep Freeze prevents permanent infection on systems. AV helps mitigate risk, but we also have had (and still do) AV signatures that will kill legitimate executables due to a bad signature in the database somewhere, so the AV can be as big a pain in the arse as the malware itself. Firewalls are blocking access outside our network. Backups are in place to restore from bare metal if need be. Honeypots in the network can help detect oddball activity. Monitoring your switches and gateways for unusual activity can help. Updates on a regular schedule helps close vulnerable pathways of infection. And diversity is your friend...sometimes a Linux system or Mac can reach an AV site for grabbing tools when all the Windows systems are crippled. Linux systems are also fantastic for pulling special tools and scanners when searching for solutions on the network. It's saved my rump a couple times when troubleshooting things.
Our particular situation isn't necessarily typical, so mitigating risk is a plan you need to create specific to your environment. But that can be said about just about any risk mitigation system.
In most cases you would just wipe and restore from good backups.
In a situation where everything is infected, the first thing to do is disconnect everything from the internet. In a typical enterprise, workstations shouldn't contain anything but profile data, so restoring those with images should be easy. If you don't have the option, and cleaning is the only option, you are in for a treat. You will want to figure out what the infection is, and the treatment options. Go slow, you will want to have everything back to normal as quick as possible, but that's where mistakes happen. There isn't anything worse than thinking you have a set of machines cleaned, only to revisit them later to find out they are re-infected.
In the case of a main file server, the best course of action is to image the server to separate disks and set aside (if you have the resources) and wipe it, install fresh and restore from your backups. Without doing that, there really is no way to know if it was rooted and won't come back to bite you down the road.
