2

How do QA teams handle browser testing of Google Chrome?

Since Google Chrome constantly updates itself, there is a challenge in limiting test scope to a specific version. While we might limit support to IE8 and IE9, Google Chrome's version number is a moving target.

Possible approaches:

  1. Freeze Google Chrome version at the beginning of the dev cycle.
  2. Allow Google Chrome to update itself during dev cycle. Develop and test against whatever version is current at that moment.

Approach #1 means everyone needs to sync up on a specific version, and edit their registry to prevent automatic updates. Furthermore, you will be releasing code for version X, while the world will have mostly auto-upgraded to version X+1.

The problem with #2 is that the browser functionality could change within dev cycles. Suppose QA completes on Monday, and code is pushed to production Wednesday. If Google releases a major automatic update on Tuesday, it could cause the code to break and no one would know until it hit production.

How do mature web development organizations handle this?

bad_coder
  • 11,289
  • 20
  • 44
  • 72
frankadelic
  • 20,543
  • 37
  • 111
  • 164

4 Answers4

1

Sites that use standard/valid HTML, JavaScript, CSS syntax etc are very unlikely to break with updates to Google (or Firefox or any non-IE browser, really). Are there any instances of a Chrome update breaking legitimately valid HTML/JS in the past?

The best advice would seem to be to make sure your site is as clean as possible, and test across all browsers. "Withholding support" just seems like it alienates your users anyway.

matt b
  • 138,234
  • 66
  • 282
  • 345
  • One example where a Chrome update changed rendering behavior: http://stackoverflow.com/questions/4991046/why-is-chrome-breaking-this-layout – frankadelic Aug 01 '11 at 18:26
1

We go with option 2 for a number of reasons.

1) We try to simulate the user experience during testing (shutting off updating would simulate a deviation from the norm)

2) Would much rather find a defect in the last round of testing than have them reported by users

3) I can't think of a time where we had a serious defect introduced by testing this way. As Matt mentioned earlier it is fairly unlikely that their updates would break important functionality.

To help mitigate some of this risk we try to use selenium for our core functional automation testing to constantly test different browsers. This doesn't catch everything, but at least will find large issues (like a missing column). It also helps to be agile about these things. We can release on any required schedule (monthly, weekly, daily if needed) so if a serious issue is found we can fix it the same day.

Steve Miskiewicz
  • 1,114
  • 1
  • 10
  • 23
1

Chrome is updated so automatically and silently the only reasonable choice is to test against the most current version. You can multiple versions ahead by running the different channels: canary, development, beta, or stable.

To see how quickly Chrome updates take place across the user base look at the last graph here.

Uriah Carpenter
  • 6,656
  • 32
  • 28
0

We simply turn off auto-update on all browsers, and have a slew of virtual machines with different browser versions.

BlackGaff
  • 7,579
  • 1
  • 38
  • 45