The Conversion Rate Optimization Process – Part 3/3

conversion rate

eCommerce

AOV

7 min. read
The Conversion Rate Optimization Process – Part 2/3
Hypothesis Development Launching a test without a hypothesis is like starting a journey without knowing...

conversion rate

5 min. read
The Conversion Rate Optimization Process – Part 3/3
Now that we have introduced you to CRO research in part 1 and formed a proper hypothesis & outlined the...

conversion rate

eCommerce

AOV

Now that we have introduced you to CRO research in part 1 and formed a proper hypothesis & outlined the needed treatment in part 2 it is time to cover test implementation, measuring tests, and ongoing learning.

We will start by zooming out from the individual test level covered in the last part and looking at the CRO efforts as a whole.

Forming a testing plan

There is one more step to do before you jump into testing, you need to have your tests prioritized. To prioritize the tests we recommend using a test prioritization framework where you calculate a score for each implementation based on expected impact & ease of implementation. The higher the total score, the higher the priority for testing that assumption:

After you have prioritized your tests, you’ll need to create a testing plan. It is very important to form a testing plan before you move on with executing to ensure that what you are testing is clear in timelines, measurements and expectations. A spreadsheet is a good choice for this.

The testing plan should include:

  • Test page URL
  • Hypothesis
  • KPIs measured
  • Estimated test duration
  • Test start date
  • The test conclusion date (Filled post-test)
  • Results (Filled post-test)
  • Confidence level (Filled post-test)
  • Link to further test info (if any). 

Related Blogs

Your testing plan instantly tells everyone reading it what are the expected outcomes, timelines, and what we expected to find. It also ensures you’ll be able to change your testing personnel and don’t run any tests twice!

Keep this document for you to review in the future and track as you move along. Because any single test can often take a long time, you might forget the original drivers behind the test. Also, over the years, it is common that your testing personnel or CRO agency to change. So, this document can also be helpful if there are any such changes taking place.

The fewer conversions you have, the bigger of a lift you will need to have to have a statistically significant test. This means that the smaller sites require more radical changes for them to be statistically detectable. This is a good to keep in mind when creating the testing plan. One of the common worst-case scenarios is being too timid and ending up with an endless stream of undecided results.

You also don’t want your testing to take a long time since, over time, cookies start to expire and seasonal changes come more and more into play. In case you’re not sure what a cookie is, you can read more about it here. Generally, it is recommended that you aim for tests that you can conclude within four full weeks. This gives you a representative sample size without too much data pollution (cookies expiring, seasonal changes, and monthly behavioral changes). 

BECOME OUR NEXT SUCCESS STORY

Learn how you can also increase your store's
eCommerce conversion rate today

Learn how you can also increase your store's eCommerce conversion rate today

You will find this tool handy to calculate your required sample size in terms of visitors you need. The required total sample size should not be higher than your total monthly visitors. That’s because if the sample size required is greater than your total monthly visitor count, then naturally, your test will take more than a month.

Your testing plan instantly tells everyone reading it what are the expected outcomes, timelines, and what we expected to find. It also ensures you’ll be able to change your testing personnel and don’t run any tests twice!

Keep this document for you to review in the future and track as you move along. Because any single test can often take a long time, you might forget the original drivers behind the test. Also, over the years, it is common that your testing personnel or CRO agency to change. So, this document can also be helpful if there are any such changes taking place.

The fewer conversions you have, the bigger of a lift you will need to have to have a statistically significant test. This means that the smaller sites require more radical changes for them to be statistically detectable. This is a good to keep in mind when creating the testing plan. One of the common worst-case scenarios is being too timid and ending up with an endless stream of undecided results.

You also don’t want your testing to take a long time since, over time, cookies start to expire and seasonal changes come more and more into play. In case you’re not sure what a cookie is, you can read more about it here. Generally, it is recommended that you aim for tests that you can conclude within four full weeks. This gives you a representative sample size without too much data pollution (cookies expiring, seasonal changes, and monthly behavioral changes). 

Also, you can use tools like visual website optimizer, and A/B calculators to calculate the A/B test duration or have a trusted agency run all of this for you.

Now, you are ready to launch the tests!

Executing your testing plan

 

Now that you are executing the testing plan and are ready to run your tests, you need to make sure you write, design and develop your landing/testing page appropriately. It is good to expect 2-3 rounds of copy editing, design iterations and quality assurance until you get the exact page you’re ready to test. 

At this point, your test pages should be bug-free and work perfectly. Even one silly mistake in the testing implementation can screw up all of the hard work you have done so far. This means that you need to make sure your testing variations are thoroughly tested before you launch them.

Moreover, a common mistake is not testing the pages for bugs AFTER you launch them. We’ve heard of numerous stories where pages worked perfectly before launch, and then 2-3 days into a test, the page stopped functioning as expected. Take the necessary measures to make sure this isn’t you. Otherwise, it’ll most likely lead to 2-3 weeks of lost testing time (and a lot more in lost revenue!).

What you can do in this case, is set up reminders for yourself 2 days, 5 days, and 10 days into your testing period. The reminders are to double-check that test pages are working exactly as expected throughout the testing period. You can also peek into the data to see that conversions are tracked properly, but don’t draw premature conclusions based on very little evidence.

Learning

 

The last step, which is also the most important step long-term is learning. Now that you have run your test, you got your result, and now there is an opportunity for tangible learning that you should not just overlook. When done right CRO is not just optimization. It is also a powerful method to learn more about the users and user behavior. In many cases, startups have changed their entire business focus based on learnings gathered from A/B testing and CRO research.

Remember: you are running the tests based on assumptions, which means that ideas and opinions don’t matter. We have to be able to validate or invalidate them based on actual data. And this is the time for you to validate or invalidate the assumptions! 

By analyzing and learning the changes, you will have a clear picture of your users, what works for them and what doesn’t, and then utilize these findings throughout the site. This is a great example of the Build-Measure-Learn loop, where you utilize what you have learned.

Understanding behavior in the testing variation

There are two resources to check when you are analyzing the variant and trying to learn from it: Google Analytics (impact on the whole) and heat mapping tool (behavior on that variant page). 

With Google Analytics, you can discover the different testing groups and what kind of actions they took on the page and how they moved through the website. This can help you identify possible bottlenecks and patterns amongst multiple “winning” variations or “losing” variations. 

With your heat mapping tool, you can review heatmaps and session recordings of winning and losing variations of your test, which can help you understand more about your users and help guide you for future tests that you’ll be implementing shortly. 

CONCLUSION:

That’s it! 

The steps outlined above and in parts 1 & 2 of this 3-part series are the steps of a perfect conversion rate optimization process. The keyword to emphasize here is process. On the surface running a couple of tests can appear quite simple but there are a lot of elements to consider while building and implementing an A/B test from start to finish.

We hope that we’ve outlined the tools and methods to help you run A/B tests on your own. However, if you’d like some help optimizing your eCommerce store & want to take it off your own plate (handing it over to experts), you can apply to work with us. Since 2015, we are doing the exact things for 100’s of clients. You can apply for our support here: <a  href="https://www.crogurus.com/work-with-us" style="text-decoration: underline; color: #0288D1;" target="_blank">www.crogurus.com/apply</a>

The Conversion Rate Optimization Process – Part 3/3
14 min. read
The Four Apps You Need To Run A Successful Shopify Store
Shopify was built with simplicity in mind and offers only the basic functionalities needed for running...

Shopify Optimization

3 min. read
How To Optimize Your ECommerce Store For Conversions – Top 10 Fixes You Can Do Today
I am going to share some of my...

eCommerce

12 min. read
The Conversion Optimization Process – Part 1/3
Let me guess.. the reason you visit this blog is because your e-commerce store is not generating enough sales...

conversion rate

eCommerce

BECOME OUR NEXT SUCCESS STORY

Learn how you can also increase your store's eCommerce conversion rate today

Learn how you can also increase your store's eCommerce conversion rate today

Related Blogs

Now that we have introduced you to CRO research in part 1 and formed a proper hypothesis & outlined the needed treatment in part 2 it is time to cover test implementation, measuring tests, and ongoing learning.

We will start by zooming out from the individual test level covered in the last part and looking at the CRO efforts as a whole.

Forming a testing plan

There is one more step to do before you jump into testing, you need to have your tests prioritized. To prioritize the tests we recommend using a test prioritization framework where you calculate a score for each implementation based on expected impact & ease of implementation. The higher the total score, the higher the priority for testing that assumption:

After you have prioritized your tests, you’ll need to create a testing plan. It is very important to form a testing plan before you move on with executing to ensure that what you are testing is clear in timelines, measurements and expectations. A spreadsheet is a good choice for this.

The testing plan should include:

  • Test page URL
  • Hypothesis
  • KPIs measured
  • Estimated test duration
  • Test start date
  • The test conclusion date (Filled post-test)
  • Results (Filled post-test)
  • Confidence level (Filled post-test)
  • Link to further test info (if any).