Welcome to the wild world of A/B testing emails! If you are responsible for email marketing, you know that every subject line, word choice, and link placement are variables that affect your email performance. Not to mention, factors such as send time, day of week, month, etc.
The crazy thing is, the performance is all measured, right down to the last click.
If you’re like most marketers, you’re probably spending dozens of hours testing different variations of the same email to:
But there’s just one problem with A/B testing your emails. You generate a lot of data, especially if you have a large email list.
Your marketing automation tool isn't likely the best tool for data exploration and analysis. Therefore, you are going to want to export the data, open the CSV online, and use a friendly spreadsheet format for analysis.
What is the best way to Analyze it? Well, let’s have a look.
I pitched this idea to my close friend Will Andrews, who’s in charge of Product and Marketing at Gigasheet. He collaborated with me to build a dummy A/B email testing dataset – where we together came up with:
Here’s a sneak peek at our dataset:
Now, let’s analyze it!
I work as a freelance content partner for Gigasheet. But that doesn’t mean I hate Google Sheets or Microsoft Excel.
In fact, I love them!
To conduct my analysis, I loaded my spreadsheet to Google Sheets, which you may find here.
If you’re like me, you already know how complex Google Sheets is. From giving you the ability to play with spreadsheets and perform calculations to advanced features like conditional formatting, data validation, data filtering, and sorting, data visualization, and more, Google Sheets is incredibly complex.
Every time I had to apply a filter or formula – I had to look it up via “Help.”
While you’ll soon get the hang of the platform (know all about its features), it takes time. Google Sheets has a steep learning curve.
First, I wanted to sort my data by email variations. I’ll admit – it was simple – all I had to do was create a filter and sort the data in the email variation column by “Sort A -> Z.” You can also hide or unhide column groups in Google Sheets easily.
But then came the hard part.
I wanted to find out – which version of my email had the highest open and click-through rate. I thought of applying a filter – but if I went that route, I’d have to count open and click-through rates for every email variation – which is just too much work.
Well – you can always apply grouping – but when I tried doing it in Google Sheets – I found out that it was one of the hardest things on the planet. Using Google Sheets to group your data is like getting a toddler to eat their veggies – it’s HARD.
On top of this, did you know that – Microsoft Excel and Google Sheets have their own limitations? At present, Google Sheets can only process 10 million cells. Excel’s row and column limits stand at 1,048,576 and 16,384 respectively.
Also, if you try to access a large spreadsheet file using Microsoft Excel or CSV, it may result in your PC crashing or browser/software freezing. I have been known to cause Excel to crash!
That’s the reason we encourage people to use Gigasheet.
As a freelance writer, I keep my clients limited. And the reason I work with Gigasheet is that I love the platform. No matter how large your spreadsheet file is or whether it has billions of rows, Gigasheet operates at cloud scale – so neither your browser will hang, nor you’ll find it tough to play around.
And the best part is – Gigasheet has so many advanced features like data filtering, sorting, grouping, validation, and more – which will help you get your hands on the exact information you need. Insights are easy with Gigasheet!
Let’s try it out.
Let’s perform the same operation that we were trying to do above. I want to find out – which version of my email has the highest open rate. I loaded my spreadsheet to Gigasheet – and added two layers of column grouping:
Here are the results:
Here’s the story data told me:
In terms of open rate, the second variation won – with an open rate of 55.27%. Whereas, the fourth variation had the lowest open rate, i.e. 45.85%. This helped me understand that the second variation has the best subject line.
Similarly, I looked at the click-through rate data by applying the grouping:
Here are the results:
You’ll find that there are blank cells in my file – that’s because if an email was not opened, then actions like click-through, responding back, booking a meeting and unsubscribing were not taken. So, I have kept those cells blank.
Upon analyzing the data, I found out that –
Here, Variation 1 is the winner with a CTR of 29.18%. Whereas, Variation 5 has the lowest CTR of 24.22%. Variation 4 had the lowest open rate; however, variation 5 has the lowest CTR. So maybe Variation 5 doesn’t have an up-to-the-mark email copy.
Similarly, you can add or modify your grouping to find out information like which variation –
In today’s world, we generate a lot of data. It’s manually impossible to process and make meaning out of it – that’s exactly where Gigasheet saves the day.
Next, I wanted to identify the emails that unsubscribed – so I applied this filter:
I found out that 132 people unsubscribed. Find it at the bottom of the screenshot below.
Like I said before, the amount of A/B test data you generate depends on the size of your email list and the number of people you choose to roll out emails to, if you run normal email marketing campaigns, you’ll be generating a lot of data. And you’ll need a platform to make meaning out of it.
In my opinion, there’s no other platform better than Gigasheet when it comes to processing and analyzing large data.