0

I recently worked on refactoring a class for my company's web site which in essence assembles a user profile. Say the old class is called User::Profile::OldImpl and my refactored version is User::Profile::NewImpl. The two classes have exactly the same public interface and are expected to behave identically. The new class is just much easier to read and more efficient--or such was my intention.

I changed the place where the profiler is instantiated from this:

profile = User::Profile::OldImpl.new(...)

...to this:

profile = User::Profile::NewImpl.new(...)

During code review, my reviewer flagged this line and told me to add an A/B test here. By this I suppose he means to sometimes use the old class and sometimes use the new class.

I'm mightily confused. I understand "A/B testing" to mean showing different versions of a web page to end users and tracking which version generates the best response, but my code is meant to be invisible to end users--it doesn't touch the view at all. I've searched around and haven't been able to find a discussion of A/B testing that relates to backend code like mine, and I can't imagine why my change merits an A/B test when the great majority of checkins don't.

The review was submitted near end-of-day Friday, and I'm obviously going to chat with the reviewer first thing tomorrow, but I was hoping to first learn from the SO community whether A/B testing can legitimately be applied to a change like mine, and if so, how.

Sean
  • 29,130
  • 4
  • 80
  • 105

1 Answers1

0

You reviewer is worried that despite no visible changes, your implementation may affect some important business metrics. What he wants you to do is to prove empirically that there's in fact no change from the user perspective, i.e. that your A/B test is actually an A/A test. I'd ask him what KPIs he has in mind.

Igor Urisman
  • 717
  • 1
  • 6
  • 22