2

I am very new to this subject and was hoping someone could shed some light on it. I am working on creating a corporate network that will obviously have multiple servers and multiple workstations.

Let's say a new version of Adobe Flash comes out. I would think that you would want to test this update in a test environment before "pushing it out" to the servers and workstations.

How do you guys go about controlling, testing and then pushing the application updates out? (i am not talking about windows updates). Do you use a 3rd party sysadmin tool? Home grown software?

Any info will greatly be appreciated :)

user145133
  • 123
  • 2
  • There could be a whole book or site on this... actually there's a book (look for limonicelli's "The Practice of System and Network Administration") and a site (Serverfault) on the topic. Its really far too vast a topic, and one with so many approaches to cover in a simple answer. – Journeyman Geek Jul 10 '12 at 14:44
  • My advice is do not post it con Serverfault because the question is too broad and will get closed (been there -_-). Go for books as Journeyman suggested. – Rhyuk Jul 10 '12 at 14:48
  • *Personally* I 'test' updates on my own PC first, but it's not long before other users have 'tested' it... Such are the pains with not using group policies! ;) – HaydnWVN Jul 10 '12 at 15:04
  • If you can, rolling out your own scripts can be a valuable skill. But be sure to check what's there already, books and websites and tutorials and ready-for-you scripts; they'll probably exist en mass out there. For instance, Haydn mentioned group policies. Here's an approach: 1) Disabling auto-update. 2) Test it. 3) Provide the .msi installers on a network share. 4) Run a script at shutdown (or when appropriate) that checks which msi files aren't installed, and run a silent install of them. 5) You might want a way to report back to the server so you get an overview, or automatic log uploads. – Tamara Wijsman Jul 10 '12 at 15:04

3 Answers3

2

Microsoft's SCCM and various other product will do this. However, they let you do the specific task: Install the software. The big question is how do you coordinate this?

In "The Practice of System and Network Administration" there is a chapter that recommends the following methodologies:

  1. "one, some, many" -- Upgrade your own machine and test for a few days. Upgrade a few more (say, the other sysadmins on your team). Then roll out to "many": larger and larger groups.

  2. "canary" -- Upgrade one every [period of time] until done.

  3. "exponential" -- Upgrade 1, then 2 more, then 4 more, then 8 more. The group size doubles each time.

  4. "risk-adverse last" -- Divide the organization into groups and do the most risk-accepting first, the more risk-adverse last. For example, there may be one group that prides itself on being cutting edge and will volunteer to go first (the IT department, the engineering department). There may be a group that is very suspicious of upgrades and they go last (accounting dept, the executives, etc.) Smaller groups should probably go first too.

No matter how you group the upgrades, upgrades should be tested first, and.

After each "group" of upgrades, do a series of tests. If any tests fail, or if problems are reported, stop doing upgrades. Revert back to the previous release if possible (or safe).

The upgrades should not start until you have done your own tests. For example, in a lab or on your own machine. More structured tests would include trying the upgrade on one of each kind of machine, one of each release of the OS, and so on. Tests should include starting and stopping the software, as well as running its major functions (since you mentioned Flash: try playing a video, running a flash game, and so on). It is good to keep a wiki page that documents what combinations where tested and what tests you ran. The next time you upgrade this package you have a good list of tests to use. If a problem is reported during upgrades, add a test to the list to prevent that problem in the future. Since you mentioned Flash, I recently found a problem with the Weight Watcher's food tracker app and a certain version of Flash. We added the URL for that app to the list of tests and now we know that new Flash upgrades must be tested with that before we release it.

Between each "group" of upgrades, pause for some amount of time to see if errors crop up. Whether this is a day or a week depends on many factors: is this a big change? were the previous group upgrades successful? Monitor your Helpdesk tickets for reports of problems related to the upgrade. If you have full-time helpdesk attendants, keep them apprised of what upgrades are in progress so they are on the lookout for problems.

Whether you use "one, some, many" or other methodologies depends on many factors. "One, some, many" is good in smaller environments. "Exponential" is good in a large desktop environment where hundreds of machines are centrally controlled. "Risk-adveres last" is good when you can divide your users into specific groups that have different "personalities". "Canary" is used on web farms and grid computing where you have hundreds or thousands of machines all with the same configuration.

The most important thing is to take good notes. If you had to do a good upgrade once, you'll have to do more upgrades in the future. You want the process to become repeatable and keeping a list of tests performed is key for that. The next time you do a similar upgrade there will be less thinking you have to do, which means fewer mistakes ("oops, I forgot to test blah-blah-blah") and it will go faster. In fact, if you just keep basic documentation then you can delegate this to that new, junior, sysadmin you've hired. He or she can repeat your process, add to it, and improve it. You can focus on training them and checking their work. Meanwhile you can work on other projects.

TomOnTime
  • 7,945
  • 6
  • 32
  • 52
1

There are tools to assist in this like Microsoft's SCCM, Active Directory Software Deployment, Altaris, LanDesk, etc. There are a million and one ways to push out updates, but business that follow best business practice and don't make their users admins use at least one of these.

As for testing, I ususally push the update to my own machine and a small test lab first. Poke around for a few minutes, then push it to a select group within the IT department that know that they are part of my early release schedule. Then, if there are no problems I push it to everyone.

MDMarra
  • 100,734
  • 32
  • 197
  • 329
  • Awesome. Thanks for the info. Which one do you use? – user145133 Jul 10 '12 at 18:58
  • I've used AD software deployment in the past, but it's the least capable solution. I'd recommend SCCM to anyone looking that's in an all MS shop. The place I'm at now uses Landesk, but I don't have anything to do with client software deployment in my current role, so I can't vouch for it. – MDMarra Jul 10 '12 at 19:03
0

if you're looking for a lightweight tool for scripting system upgrades and managing different profiles - take a look at wpkg. it provides you just a framework but at the end you have to find out the upgrade/quiet install/uninstall syntax by yourself.

once you have it in place you can have few virtual machines - members of a test group - where you test new versions before making them available to all the users.

sites like itninja [former appdeploy] or wpkg.org can be source of unattended installation recipes for different software.

pQd
  • 29,981
  • 6
  • 66
  • 109