Resources/Get more here/Testing (A/B and MVT)

Set the statistical confidence level for tests

David Kavalsky
posted this on June 02, 2011 09:38

LiveBall can compare the conversion results of different Creatives and determine which one will be more successful based on a predetermined confidence level.  Within your LiveBall console you can define this level between 80% and 99% confidence at your preference.  

How to

To set your preferred confidence level, hover over the “Settings” tab in your LiveBall console, and select “Configure Site” from the drop down.  At the bottom of the page, you will see a drop down menu labeled, “Confidence.”  This is where you can select the level of confidence you would like to achieve before declaring a winner of a given test between creatives.

 *TIPS:

  • Selecting a higher confidence level may require more respondents (traffic) to achieve confidence and therefore takes longer to achieve than lower confidence levels. 
  • Selecting a higher confidence level will result in more accurate test results (a higher level of confidence means the margin of error is smaller on the results).
  • You can view your test confidence in the "traffic source performance" tab for any of your traffic sources—check out the "creative summary" gauge for the margin of error, and check out the "confidence interval" to view the test results at both 80% & 95% confidence.