If we want to maximize average revenue per user with a/b testing, we need to be able to measure the dollar amount of the purchase. When simply looking at conversion rate, a variation might show 10% conversion rate versus another variation that only has 3% conversion rate. However; in a pool of 100 users; 3 purchases at $10 each is much better than 10 purchases at $1 each. Rather than sending a single goal event, for example "purchaseMade", from each user, will the API support sending multiple goal events from a single user? So when a user spends $10 in the app, can we send 10 x "1dollarSpent" events and see it reflected in the a/b results? Thanks.
Hi Watson, The standard errors are not calculated correctly for multiple events. This may inflate the false positive probabilities and cause incorrect decisions. The method of trying to record many conversion events won't work. Our system is constraint that we have to have less conversions than views. One thing you could do, is translate your monetary conversions into Boolean conversions on the client and then use a multi-goal test to get better info. To do this, you'll have to keep track of the amount purchased on the client and then record a specific event when they reach a threshold. For example if you have the following events: userPurchasedSomething: record if user has bought anything userSpentMoreThan1: record if user spent more than $1 userSpentMoreThan5: "" "" $5 etc. Given these events, you can setup a test that tracks these goals. It won't give you an exact revenue of each variation, but it will give you more insight than what you could before. Given these events, you can setup a test that tracks these goals. It won't give them an exact revenue of each variation, but it will give them more insight than what they could before. Hope it helps. Thanks.