Interdependence Software (servers schema of a company)

Question

In the web world, so many little things can break a website (updating a library, a code, etc..).

We've got in our company this problem to not check enough what can happened after doing an update).

This topic is there to explain you what we're looking: a software / your advices to check easily all the impacts that can provoke a new code / a new update of a library / etc..

Here are two examples:

Example 1:

We've installed NewRelic in one of our server, however, we didn't know that NewRelic (apparently) was including a lot of PHP files for each requests.

This is what happened: the PHP limit max included file has been reached and some pages was crashed (E_WARNING error).

Example 2:

We've changed the kind of replication between 2 MySQL Server (from Master -> Slave to Master <-> Master).

This is what happened: we used Percona and a misconfiguration led to the banning of the two servers themselves (due to a large amount of SQL query).

Our question:

We'd like to detailed as much as we can our current structure. Then, we would like to rely all the servers, the library used for each servers, the specificy of our homemade code, etc.. and use this data when we want to upgrade anything.

We could say "I'm going to update a small PHP code there on this server". And the software could help us to know all the possible troubles / impacts.

Answer 1

Very easy : It's called a test or staging environment.

You have it configured exactly like your production environment, then you deploy to it in the same way that you would deploy to production, and then you test it extensively.

There is no analytical way to do this; you have to do it. /edit - to expand on that. There's no way that software alone would be able to do this, especially with your own products in the mix. You would have to build a rule-set that was based on a detailed reading of every set of release notes, and probably the bug databases, for every product involved - assuming that they were public and complete, which is not something to count on, ever. Thus, the only real test is to perform the planned upgrade and test functionality - preferably including something like a production load.

This is the exact same reason that you do backup/restore testing and DR testing - you won't know what you missed, or what was configured improperly, until you're forced to do it. And it's better to do it under a plan, as a test, then winging it in production.