May 3

Pros and cons of the popular A/B testing tools in the market


If you are even an occasional follower of popular conversion optimization and related blogs, you know that testing is a prominent requirement for your web business’s success. A customer base is a relative term, and it always unique in different time spans. You might be lucky to have a recent modification performing well without testing, but over the long run, you might not feel always so lucky.

A/B testing isn’t something based on intuition or being lucky with the guess. It’s a science that works on data analysis and statistical calculations. Fortunately, you don’t have to worry about the data analysis and calculations by yourself, because there are powerful tools available to work for you.

Here, I am going to discuss the pros and cons of some popular A/B testing tools in the market. The tools mentioned here are very much successful and fully supported by the dedicated teams. So, stay tuned for our review of these popular tools at your disposal.

1. Google Analytics 

Google Analytics

Google Analytics is one of the free tools on our list. It is the baseline of all the business data and internet data analysis. Being one of the most used tools, Google Analytics offer a feature called “Content Experiments” which provides the split testing interface in the program.

Pros of Google Analytics include:

  • It is a free tool by Google
  • Simple interface with no cluttering
  • Advanced reporting system
  • Easy tracking of business targets with “Goals” feature
  • Quick and efficient multi-armed bandit testing

Cons of Google Analytics include:

  • Provides a wealth of data, but you need to be a proficient analyst to use the data
  • Does not have multivariate testing and other testing options
  • Lags while using on dynamic pages
  • Absence of visual editor makes it difficult to create variations

2. Optimizely


Optimizely is one of the first choices for business owners who are seeking a premium testing tool. With simple installation and efficient visual editor, it is highly user-friendly even for the rookie users. It provides a cost-effective entry point to the A/B testing world with it’s cheaper plan starting from $17.

Not just the installation and user interface, Optimizely’s A/B testing software is also very good. It provides the options for Split testing, multivariate testing, and mobile testing. You can upgrade your plans to the higher level anytime you wish to.

Pros of Optimizely include:

  • Cheaper option for entry level users with affordable plans
  • Easy to use even by a non-technical person
  • Multiple testing options- Split, multivariate and mobile testing
  • You can create custom user segments that can be reused in experiments
  • Option to unlock advanced features by upgrading the plan

Cons of Optimizely include:

  • Cannot integrate outside data sources
  • Absence of bandit or adaptive testing makes it difficult to judge the results
  • There’s no experiment version control
  • Multivariate testing is not included in the basic plan. You have to upgrade to $359 per month plan to unlock Multivariate testing.

What users have to say about Optimizely:

“Because of its simplicity and usability, Optimizely often gets a reputation as a “lightweight, introductory tool” to use until you are ready for a more enterprise product. It’s marketer friendly. It’s developer friendly. It’s analyst-friendly. It’s pocketbook friendly. Overall, this is a very compelling product and company.”Ryan Garner, Co-Founder, Clearhead

3. Visual Website Optimizer-

Visual Website Optimizer

Visual Website Optimizer or so-called VWO is a feature packed testing tool which became extremely popular in the last two years. It almost provides everything that is there in the Optimizely, except for the starting price is $49 per month. Additionally, VWO provides seven different testing options which include:

  1. A/B Test
  2. Multivariate Test
  3. Split URL Test
  4. Targeting
  5. Usability Test
  6. Conversion Tracking
  7. Heatmap and Clicks

This why VWO’s plan is much costlier than the Optimizely’s. The inclusion of usability testing tools and heatmaps provide an USP to the program. If you receive some serious traffic and want a beginner to intermediate level tool, VWO could be good option.

Pros of VWO include:

  • Feature packed tool with multiple testing options
  • Provides usability testing
  • Easy to use interface
  • Idea generation tools
  • Multiple targeting options

Cons of VWO include-

  • It is a bit costly choice for first timers
  • Absence of multi-page testing features
  • Lags in real-time support options as compared to other alternatives
  • URL targeting is a bit complicated

What others have to say about VWO:

“Visual Website is better for beginner and casual testers, with extra built-in tools like click heatmaps to help you find test ideas”- Richpage

4. Unbounce


Unbounce follows a completely different approach than all of the alternatives in our list. It has established its niche by solely concentrating on the landing page optimizations. Despite its limited niche, Unbounce has managed to make itself a leader in the domain of A/B testing tools.

It performs like a unique destination for pre-designed, customizable, and well-optimized landing page templates for the subscribers. Each of the Unbounce’s landing page templates is designed by the Unbounce team taking into account all you need for a conversion-friendly landing page.

Not just the templates, it also offers the A/B testing tool that has been designed specially to test your landing pages. You can leverage it’s few but effective metrics for data interpretation. If you are a business startup and truly into the landing pages, probably Unbounce would be the best and quickest option for you.

Pros of Unbounce include:

  • Pre-designed landing pages makes it easy for the rookie users
  • Solid Split testing for Landing pages
  • Quick variations can be prepared using the customizable templates
  • Simple to use
  • Can be integrated easily with other marketing and testing tools.

Cons of Unbounce include:

  • Limited to A/B testing for landing pages only
  • Could be a costly investment for limited features
  • Lags other testing options
  • Lags usability testing

What users have to say about Unbounce:

“From a small business to SME, Unbounce does a great job building and managing landing pages for your marketing campaigns. If you are looking to build a landing page with no coding skills, go to Unbounce. Even a 6-year-old can learn how to build a slick looking landing page.”Tomas Kolafa, Co-Founder, Marketer, Growth Media Co

5. KISSmetrics


Well, KISSmetrics is a big name in this domain and probably there is no one in the internet marketing who has not heard of it. This well-known tool provides the excellent interpretation of the human side of the data and reports on the basis of data from real people. It tracks the interactions of different users on your website to prepare a well define insight to your site data.

Probably a high cost of $ 150 per year would not be a big deal for such a useful program. The usability tracking helps you to track the users on the site and let you identify the leaks in the funnel by pointing out the entry point, exit point, and interactions in the mean time.

Pros of KISSmetrics includes:

  • Flexible reporting system
  • Simple user interface
  • Usability tracking of visitors on the site
  • Can trace real data from real visitors on the site
  • Integrates well will other KISSmetrics software

Cons of KISSmetrics includes:

  • Extreme focus on Usability testing only
  • A/B testing is comparatively weaker than the other alternatives
  • Does provide demographic and location-based data by default.
  • Provides a wealth of data that needs expert level proficiency to interpret

What users have to say about KISSmetrics:

“Our organization uses Kissmetrics for business intelligence analytics and user tracking. Currently, we’re using it for building analytics for a multi-million user base and Kissmetrics has never run short for our purposes. The BI department uses it for gathering useful data and the dev team has always felt comfortable when working against Kissmetrics. ”Javier glesia Sabio, Software Engineer, Paradigma Tecnológico

6. Crazy Egg

Crazy Egg

Like Unbounce, Crazy Egg has also built its niche by focusing on click tracking. The click tracking helps the marketers to understand user behavior on the site, and lets them identify exactly where users are clicking when on the site.   It provides an easy look into the collected data by representing the click tracking data in different formats and visual graphics. If you are a rookie user, you won’t find a hard time understanding the reports. With a starting package of just $9 per month, probably it is the cheapest premium alternative in the list.

Pros of Crazy Egg includes:

  • Click tracking makes the usability tracking insightful
  • Visual reporting system makes it easy to understand
  • Multiple reporting formats as per the requirement
  • Cheaper than most of the alternatives in the list

Cons of Crazy Egg includes:

  • Limited set of features even for usability tracking
  • Only click tracking
  • Limited testing options
  • Focused on click tracking only
  • Lags testing for pages with dynamic value in the URL

What users have to say about Crazy Egg:

“Crazy Egg is great as a heatmapping tool – If you want an affordable way to see click, scroll, and positioning data for real visitors on your website, Crazy Egg is a good option. It can’t compare with tools like Google Analytics, but that’s not how it is designed to be used.”Albert Wavering, Analytics Architect, Blue State Digital

7. MockingFish


Our list would not be complete without mentioning the home team’s offering. MockingFish, probably not so popular like the other alternatives discussed here. The reason is, it’s a new entrant in the domain and has just started its journey. Being the youngest contestant makes it so important for the program to prove its significance and efficacy among the well-established programs.

Probably this is the reason why it is the cheapest and most accessible of all here. It provides the massive trial offer by providing a full one-year subscription to the premium features for free. It is not just an A/B testing tool, but also an effective usability tracking tool through heatmap tracking.

Talking about the functionalities available, you can do: 

  1. A/B split testing
  2. Mobile testing
  3. Click heatmap tracking
  4. Scroll heatmap tracking
  5. Mouse movement tracking
  6. Audience targeting
  7. URL targeting- substring, simple, expression, and Dynamic matching etc.

With its easy integration solutions for various platforms like WordPress, Drupal, PrestaShop, Joomla, Magento etc, it becomes so easy to apply the code snippet to the website’s code.

Pros of MockingFish include:

  • Impressive trial period for one year- not offered by any other alternative
  • Multiple testing options- A/B testing and heatmap usability testing
  • Tracks data from real users
  • Multiple heatmap tracking options- List Heatmap, Envelope Heatmap, Insight Heatmap, Click Heatmap, and Scroll heatmap
  • URL and Audience tracking- Not available even in some popular programs – Easy to understand reports – User-friendly interface
  • Backed by dedicated support team

Cons of MockingFish include:

  • Lags Multivariate testing in the trial period
  • Provides so much of data, but some of them need pro level expertise to interpret
  • Still very young. So, might not find an active community of users
  • You might have to depend on support team to clarify your queries (you are extremely welcomed to ask anything)
  • Lags specialized funnel reporting. Though funnel testing can be done with URL targeting, you have to interpret the data by yourself to check the funnel performance.

What users have to say about MockingFish:

“Mockingfish- Heatmap and A/B Testing is a great module, it provides complete knowledge about customers behavior. It provides statistical data generated according to the customer’s behavior. It helps to improve the areas where the customers visit very less or do not visit at all ”Kendra, certified user, Website owner


Post A Comment