Resources/Help Topics/Testing (A/B and MVT)

Get started with A/B testing

Jessica Collier
posted this on Jan 24 16:03

Before We Begin

Start with a test plan — know what you want to test and why.  If you are new to testing, you can download a copy of ion’s “Guide to Online Testing” for more information on formulating your test strategy.

Let’s Get Started!

Engage Your Creatives

To begin A/B testing, you will need to have two or more engaged creatives within a single campaign. Once you have your creatives ready, setting up the actual A/B test only takes about one minute! It’s that easy, just follow these instructions for opening a traffic source and splitting your traffic across your creative variations. 

  1. Navigate to the creative management screen for each creatives and click the drop-down in the top right corner labeled “MODE”
  2. Select “Engaged” from the drop-down menu and then click the blue refresh button immediately to the right
  3. The pages within this creative should now be highlighted in green

Open A Traffic Source 

Once your creatives are engaged, you can now open a traffic source (a unique URL) that will split traffic among your engaged creatives. Opening a traffic source both takes your creative live by setting up a unique URL, as well as gives you the opportunity to split traffic across two or more creatives in an A/B test.

  1. Navigate to the campaign management screen
  2. Click the orange “traffic sources” tab just above your list of creatives
  3. Click the green plus sign on the “traffic sources” tab to open a new traffic source
    1. Select the subdomain you would like to use with the URL you are creating
    2. Add a unique identifier for this page in the text box immediately to the right of the subdomain drop-down menu
    3. Choose how you would like to track visitors
      1. This will show in reporting how traffic is driven into this URL
      2. Choose a category and a sub-category
        1. If there is not a sub-category that you would like to use, you can add a custom sub-category by clicking the button immediately to the right
    4. Select one of the creatives you would like to receive traffic from this URL
      1. Once selected, the thumbnail of the creative you chose should be highlighted in green
    5. Click Save

Split Your Traffic

After creating your traffic source, you will be taken to the traffic source management screen.  From here, you will be able to set up your newly created URL to split traffic between multiple engaged creatives.

  1. To the left of each creative in the traffic source management screen, you will notice sliders for each creative
    1. These are weighted sliders that well split traffic based on the weighted sliders
  2. To split traffic, move the slider to the right for a creative that is not set to receive traffic from your URL
    1. Note: Only creatives highlighted in green are set to receive traffic from your URL
  3. If your sliders are all in the same position, then traffic will split evenly across the green creatives
    1. You can have your traffic split in any way you would like
    2. The percentage of traffic for each creative is represented in a pie graph labeled “Traffic allocation” to the right of the list of creatives
  4. Once your sliders have been set as you would like, any traffic that lands on this URL will be split as specified by your sliders
    1. Behind your single traffic source URL you can route the traffic across as many creative variations as you wish

Optimization

LiveBall allows you to automatically optimize, by sending traffic to your winning creative once statistical significance is reached, or you can manually monitor your test results without auto-optimization. 

  1. In the top right corner of the traffic source management screen, you will see a button displaying the word “automatic” just below the word “OPTIMIZATION” 
  2. By default, LiveBall will automatically find the better performing creative and remove the lesser performing creative from the traffic split
  3. LiveBall determines which creative is performing better based on a statistical confidence interval that is set in the “Configure Site” screen
    1. You can find more information on statistical confidence here http://support.liveball.com/entries/20170356-setting-the-statistica...
  4. If you would like to manually control when a creative is removed from the test, simply click the button below “OPTIMIZATION” and it will switch over to “manual”