Ionso Sheinberg
posted this on July 26, 2010 11:20
For each curve, *up to 5* points (see notes 1 and 2) are labeled with markers:
- The start of the 95% confidence interval. This is the left-most point on the curve that is labeled. It displays the conversion rate where the 95% confidence interval starts and if you hover over the point, a tool tip appears with this info. See note 1.
- The start of the 80% confidence interval. This is the 2nd point from the left that is labeled on the curve. It displays the conversion rate where the 80% confidence interval starts and if you hover over the point, a tool tip appears with this info. See note 1.
- The mean (or average) respondent conversion rate that has been observed. This is the “center” point on the curve that is labeled, and a tool tip informs you of this if you hover over that point.
- The end of the 80% confidence interval. This is the 1st point to the right of the mean that is labeled on the curve. It displays the conversion rate where the 80% confidence interval ends and if you hover over the point, a tool tip appears with this info
- The end of the 95% confidence interval. This is the 2nd point to the right of the mean that is labeled on the curve. It displays the conversion rate where the 95% confidence interval ends and if you hover over the point, a tool tip appears with this info
Key takeaway facts about the curves displayed on this gauge:
- There is a true conversion rate associated with respondents who visit a specific creative. It is estimated with the observed number of converted respondents who visit that landing experience.
- These confidence curves are a graphical representation of the probability (i.e. odds) that the true conversion rate falls between two values on the x-axis. That is, if you locate the start and end of the 95% confidence interval, then the curve indicates that we are 95% confident that the true creative conversion rate falls into this range of conversion rates on the x-axis. A similar description applies to the 80% confidence interval. The width of each curve gives you an indication of the variability of the conversion rates. Thinner curves imply less variability and higher confidence that the true conversion rate is very close to the observed conversion rate. This goes hand-in-hand with larger number of respondents.
- The more respondents we have, the more confident we are in the conversion rate that observed. This is a good thing from a statistical standpoint.
- Note 1: It’s possible that the start of the 95% and 80% confidence intervals can occur at the same point. In that case, there is only one point marker displayed, and the tool tip indicates that it is a combined starting point
- Note 2: It’s possible that the end of the 95% and 80% confidence intervals can occur at the same point. In that case, there is only one point marker displayed, and the tool tip indicates that it is a combined ending point.
- Note 3: Confidence curves are only displayed when the number of respondents is greater than zero and the conversion rate is between 0% and 100%. When all respondents have either all not converted (0%) or all converted (100%), the result is a “curve” with no variability in respondents, and this is *not* plotted in the gauge.When there is at least one conversion rate between 0 and 100%, then all creative conversion rates are also displayed in the legend. If you hover over a creative's legend entry, you will see a tool tip with the creative name, the # of respondents, the mean (average) conversion rate and the standard deviation (a measure of the variability in responses). If you click on the creative's legend entry, you will go to that creative's performance page.
- For relatively “small” numbers of respondents, the discrete binomial distribution function is used to plot the curve. The binomial distribution is used for events that can take on two outcomes with probability of success between 0 and 100%, like a coin toss. Here the probability of success is the probability that a given respondent will convert.
- For relatively “large” numbers of respondents, the continuous normal (bell-shaped) distribution function is used to approximate the underlying binomial curve.