hero image

Ep. 109 - The DIY Approach to CRO and AB Testing on Shopify with Niccoló Gloazzo

Part of the growth team for Kettle & Fire, in-charge of the CRO team and Amazon retail management side. Nicco has been working with Kettle & Fire for almost two years, starting as an external contractor and then joining the team full time. 

As a side hustle Nicco is a Shopify educational partner and he has an Italian course and agency about how to build your Shopify store, recently added as well as a Klaviyo course for Ecommerce email marketing.

In This Conversation We Discuss: (50 Characters)

  • [00:00] Intro
  • [00:41] Sponsor: Klaviyo klaviyo.com/honest
  • [02:02] Niccolo’s Ecommerce journey
  • [02:44] Success in talent marketplaces
  • [03:34] Sponsor: Avalara avalara.com/honest
  • [04:46] Niccolo’s duty with Kettle and Fire
  • [07:07] CRO vs Web Development
  • [08:30] Unbounce, Shopify, and DIY AB testing
  • [11:04] Sponsor: Gorgias gorgias.grsm.io/honest
  • [11:52] Landing page vs homepage UX
  • [12:53] Unbounce and Pagespeed optimization
  • [13:53] Prioritizing parts for testing
  • [15:14] Losing perspective and focus
  • [17:29] Sponsor: Rewind rewind.com/honest
  • [18:06] Balancing relationships among agencies
  • [19:39] Over-communicating is good
  • [21:39] Having secondary KPIs
  • [23:33] Start testing now


If you’re enjoying the show, we’d love it if you left Honest Ecommerce a review on Apple Podcasts. It makes a huge impact on the success of the podcast, and we love reading every one of your reviews!


Niccoló Gloazzo  

Clearly communicate the desired outcome that you want. I think those are the key pillars to (success). Have strong communication and be efficient in what you do.

Chase Clymer  

Welcome to Honest Ecommerce, where we're dedicated to cutting through the BS and finding actionable advice for online store owners. 

I'm your host, Chase Clymer. And I believe running an online business does not have to be complicated or a guessing game. 

If you're struggling with scaling your sales, Electric Eye is here to help. To apply to work with us visit electriceye.io/connect to learn more. Now let's get on with the show.

Sponsor: Klaviyo

Want to deliver marketing moments that last a lifetime?

Klaviyo is the ultimate marketing platform for ecommerce. With targeted segmentation, email automation, SMS marketing, and more… 

Klaviyo helps you create your ideal customer experience. 

See why Klaviyo is trusted by more than 50,000 brands, like Living Proof, Solo Stove, and Huckberry. Keep your customers coming back. Get a free trial at klaviyo.com/honest. That’s K-L-A-V-I-Y-O.com/honest.

Chase Clymer  

Hey everybody. Welcome back to another episode of Honest Ecommerce. I'm your host, Chase Clymer. And today we're going to talk about a topic that comes up all the time. 

So I'm super excited to have someone here on the brand side to chat about it. And that topic is AB Testing

And we're gonna walk through a few ways to do that with doing it yourself, when you have a smaller budget, and then when you start working with a marketing team or a marketing... An agency partner, as you will. 

So we're gonna talk about Nico. Sorry, we're gonna talk about CRO today with Nicco from Kettle and Fire. Welcome to the show. 

Niccoló Gloazzo  

Hi, Chase. Thank you. Thanks for having me.

Chase Clymer  

I think we are going to talk about Nicco as well. So (laughs) let's get into your background. 

Niccoló Gloazzo  


Chase Clymer  

Well, what's your journey here in Ecommerce and how'd you end up in Kettle and Fire?

Niccoló Gloazzo  

Yeah. Sure. So basically... actually, I started to work for Kettle and Fire almost 3 years ago now initially just as a contractor. 

So [I] purely focused on AB testing. I got in touch with Wilson via Upwork back in the days and then started to work from there. 

And then slowly after, I think around 6-8 month[s] or maybe... Yes, 8 month[s]. Then I switched [to] full-time. 

So and then I joined Kettle and Fire as part of the growth team handling more and more tasks from conversion optimization

Right now [I'm] also handling some Amazon-side... Doing AB testing even on the Amazon side, actually. And mainly focusing on the DTC-side conversion rate optimisation and Amazon Repay management side as well.

Chase Clymer  

That's awesome. And that just goes to prove that you can land some amazing clients through those marketplaces like Upwork, and Fiverr, and all that. And you've made the jump from just being a contractor on one of those places and now you're on the team. 

So that's a cool little part of the story there. I'm sure there's a lot of people out there that are on the consultant side, and they're like, "Oh. I don't know if I like those marketplaces or not." But sounds like you got a success story from it?

Niccoló Gloazzo  

Oh yes. For sure. Also, because I'm based in Amsterdam, even if right now I'm in Italy... And for me [it] was the only way to actually, get in touch with some American clients and so on. 

So I think that that opens the door also to a bunch of other possibilities as well and meet super interesting, smart people to work with and build a really good relationship with your client and then grow together.

Chase Clymer  


Sponsor: Avalara

Since 2004, Avalara's vision has been to harness the power of cloud technology to help simplify sales tax for businesses of all sizes. Avalara solutions are designed to affordably scale with businesses as they grow over time. 

Tax Compliance is not a revenue generating activity. So Avalara's technology is designed to help you manage compliance as efficiently and accurately as possible so you can reclaim your valuable time and reduce risk in your business. 

With more than 1000 signed to partner integrations, Avalara likely integrates with the ERP, Ecommerce, mobile payment and point of sale systems you use today. 

Find out how your business can be sales tax ready at avalara.com/honest. That's A-V-A-L-A-R-A.com/H-O-N-E-S-T. 

Avalara. Tax Compliance done right. 

Chase Clymer  

So let's talk about the beginning of the journey here. What did it look like back when you were just... 

When you're a contractor and you're getting started and just poking around with AB testing or and CRO for Kettle on Fire, where did you start? Had they done anything historically? I guess what does it look like when you're starting?

Niccoló Gloazzo  

Yes. So, if I remember correctly... Actually, when I started, they also had an agency that was only focusing [on] AB testing for them and did all the conversion rate optimisation apart. And then... But I think then they stopped working with them. So in that moment is when I joined Kettle and Fire and started to do AB testing. 

So back the time we're using VWO because it's a more advanced tool to do AB testing. And I think Google Optimize wasn't as good as it is right now. So we were doing mainly site wise testing. 

So basically server-side testing, where we test a bunch of different landing pages, or as well as a big modification on the website. Even though with other tools like Unbounce, you can run specific AB testing for specific landing pages. 

But then from this site wide perspective, there was really not much going on. So when I joined Kettle and Fire, we started to do some really, let's say, AB testing sitewise or so because... 

We still have our development agency involved that could provide potentially a completely different experience on site, and giving the user 2 paths and 2 different funnels to test with. So back then, I started to implement more and more [tests]. 

Testing even pricing, which is really difficult actually, to do that on Shopify... [To] get even a “Build A Box” concept [and] test it out, that's where a customer... Actually one part... 

One type of customer [would be] looking at the website and seeing just a normal website where you just add to cart and then another type of customer where, meanwhile into this specific "Build A Box" funnel, where they need to "build their box" together every month. 

So that's pretty much how the journey looked like back then. And then from there, we start to improve, improve, improve. And we switch actually, I think, a year ago from VWO --in terms of tools-- from VWO to Google Optimize, because most of the tests that we are doing on the main site are actually site-wise tests.

 So [we test the] server side and test where we require development. And so mainly [it's] our split testing, basically. So either way, I think that in the current stage, Google Optimize is more than enough, then we'll optimize to actually prove the main metrics of the site.

Chase Clymer  

Oh yeah. You mentioned a lot of really useful information in there. And I think the first thing I want to point out is you are a conversion rate optimisation specialist with an amazing track record. But you are not a developer. 

And I think a lot of people assume that it's almost one in the same, especially because it's data, it's numbers. But I do want to highlight it. 

It's something I see all the time that people don't realize that just because someone can develop doesn't mean that they can run CRO and vice versa. And actually finding someone that does both is a pretty big rarity that I have found.

Niccoló Gloazzo  

Oh yes. For sure. Also, because there's 2 different mindsets, sometimes. For sure, if you do conversion rate optimisation, it is good that you can code or at least you know some Javascript, know some HTML, CSS, if you want to do some small front-end testing that doesn't require a really skilled developer. But it's not the same. 

When you work as a conversion rate optimization, and you look at the data, and then get the developers potentially to get a new landing page design. And you find that design, change the pricing, the back-end, the query string, and so on. 

So this is not really the same and finding someone that actually has both skill sets would be great overall. But most of the time, you know, developer, it's mainly focused into the development side and not as much as in the analytical side of the CRO part.

Chase Clymer  

Yeah. Yeah, we've got a few partners out there in the ecosystem that do CRO but they'll just be like, "Yeah, you got to work with another agency to actually implement these tests and help you build out the development side of this stuff."

So oftentimes, that's how introductions happen there. Now, you mentioned something else earlier about testing landing pages on Unbounce. How does that incorporate with Shopify? How well do they play together? 

Are there any oddities of using that as like your first step of the process to kind of get some AB tests done. What's your, I guess, experience and suggestions with using something like Unbounce to [have] a more DIY approach?

Niccoló Gloazzo  

Yeah. So I'm a big fan of Unbounce because it allows you to build up our landing page pretty quickly which looks nice overall. 

And I think it's the perfect way if you want to get tested specifically on the copy standpoint, because most of the time, I would say, when you look at the landing page, the copywriting, it's really, really important. 

So if you have 2 different landing pages talking about a product with 2 different copywriting, 2 different texts, that is also going to be completely different even if you keep the design the same. 

And Unbounce is great if you want to get the small things tested or even price tested, or as well as I mentioned before, copywriting gets tested. When you work with Shopify and unbounce, it's pretty simple because Shopify allows you to get the permalink of a product. 

So basically a link that you just copy and paste on Unbounce on the product call-to-action. And when the customer clicks on that link, [they will be] redirected straight to the Shopify checkout. 

Or you can use external tools like Carthook, for example, that [is] their own checkout. So you can [do the] same thing, just copy paste the link of the product, and which redirects you straight to the checkout. 

So overall, I think Unbounce is great to get tested at the beginning, even if you have some specific concept idea or even product idea [or] you have a product in mind [or] you are trying to launch your product. 

Sometimes there's no need to have even a store, you can try to do that with Unbounce and just see how the reaction of the potential customers and then build a store afterward. 

Or if you want to test a new type of communication, you can do that on Unbounce. And when you are sure about the results, then you implement that site-wise, and you transfer the learnings on your main site.

Chase Clymer  

Oh yeah. That's a bunch of great advice. 

Sponsor: Gorgias  

What do all Ecommerce stores have in common? 

That's right, customers. 

And those customers are going to have questions.

Gorgias is the leading Ecommerce help desk with over 5500 customers on Shopify, Magento & Bigcommerce

The software is built on machine learning to optimize your support system and allow your team to save time & money on repetitive inquiries while still remaining personalized. 

What it does is it takes all of your customer insights and information, brings it into one amazing dashboard so you can solve their problem as quickly as possible. 

If you want to give Gorgias a try, visit gorgias.grsm.io/honest to get your second month free.

Chase Clymer  

So when you're talking about building out these landing pages, these are more driven by funnels. It's what you were mentioning. 

So these would be, essentially, where you're sending the traffic from, say, a Facebook advertisement, or is this something you're trying to rank on Google for SEO? And so this is different from people that are visiting the main domain.com and browsing around? [Is it] a completely different experience?

Niccoló Gloazzo  

Yeah, that's correct. You have a completely different experience. Because mainly, those landing pages are used... [They] could be either cold traffic or warm traffic, but mainly from advertisements. 

So for example, yes, as you mentioned, Facebook ads, or even some Google AdWords

 or even affiliates, potentially. If you have an affiliate and you want to give them a specific landing page only for them with their... I don't know. 

Even the images and so on, that would be a great tool to have Unbounce. So you don't need to have a completely customized page on Shopify to do that.

Chase Clymer  

Yeah, I can understand. There's also... I would point out that it is... There is something to be said that these Unbounce tools usually aren't the quickest because they're managing iterations of content. And I'm not saying they're built clunky by any means. 

They're usually pretty snappy. But if you run them through a PageSpeed thing, you're probably going to be sad about it. But...

Niccoló Gloazzo  

Oh yeah.

Chase Clymer  

...optimizing for PageSpeed is something you do after you find a winning iteration.

Niccoló Gloazzo  

Oh yeah. For sure. I agree. Actually, we were looking at some of the Unbounce pages later on and we noticed that they're not the fastest. 

But it's an initial... It's a touch point that helps you to get an understanding if the direction that you are planning to take is the right one at least. And then you transfer, as I mentioned, the learning on the main site. 

So you say, "Okay, this specific copy angle is really working well or this specific testimonial is really working well. Okay, let's try to incorporate this into the main site so that all direct traffic, organic traffic, and so on, is going to benefit from it."

Chase Clymer  

Yeah. So you're gonna then essentially be like, "Here's what we learned." And then you take it to the team and be like, "Alright, now make this work within our Shopify theme. Custom develop whatever needs to happen here." 

When you're testing pages, are you normally testing just down to the product level? Or are you also doing collection level? What are some of the things that you're testing?

Niccoló Gloazzo  

Usually it is based on the impact that these can have at the bottom line. So in terms of revenue... 

So potentially, on average, I would say, in a store, the key pages are always the homepage, the product page, collection page, and then down to cart page and checkout. So usually, it really depends where the problem is sometimes or which one is the area that you want to optimize. 

And then you focus into the specific area [or] into that specific page. So really, [it] depends. Every time is different. And then you can potentially start by building some sort of framework and estimate... 

For example, okay, if we are going to do something on the collection page, then you set a hypothesis and decide, "Okay. Why should we test something on the collection page? What did we analyze before the need to be approved and why... And also,how much is gonna cost from our development standpoint?" 

Maybe you need a new copy [of the new] standpoint, the new images and so on. So you value the return on your investment in that case.

Chase Clymer  

Yeah, I always like looking at how the funnel breaks down. And you can do this through Google Analytics. I'm curious how you guys do it. 

But yeah, you can get pretty granular in Google Analytics and start to see where people are dropping off and be like, "Oh, there seems to be a bigger drop off on our collection page than the rest of the funnel. 

There must be something going on here." And then you can do some user testing and get to the bottom of it. Sometimes you're just so close to your own website that you don't know.

Niccoló Gloazzo  

Oh yes. For sure. I totally agree, actually. And that's why sometimes, even getting help from an external agency is really valuable. 

Because sometimes you spend so much time on your website that you don't maybe see anymore, what else can be done because you keep focusing on the same thing, maybe over and over again. 

Sometimes, [you need to] get a new pair of eyes; A fresh look at your website and at your data, and at your qualitative feedback. It's really useful to get also some new ideas, new testing ideas, and new insight that maybe before you couldn't see.

Chase Clymer  

Oh, that's fantastic advice. I think getting another opinion on what you're doing... A, it can help you get rid of things like the "analysis paralysis". What they have to say is pretty in line with what you've already discovered internally with your team. 

But B, it might point something out that you guys just weren't looking at, because it just wasn' top of mind for you because you were probably focused somewhere else in the funnel or or in some other channel. 

So yeah, always getting a second opinion is useful. It's funny you say that, because it seems like lately, we've gotten a bunch of people reaching out to us, like, "Hey, We have some teams already. But we just want... Can you audit some stuff for us? What do you think?" And we're like, "This sounds fun." Strategy is the best.

Niccoló Gloazzo  

Yeah, for sure. But yes, that's totally totally the right point. Because when you spend so much time on that website and that product, you sometimes miss some opportunities that you couldn't see before.

Chase Clymer  

Yeah. Plus Ecommerce is l just rapidly evolving right now. And there's so many cool, new tools that are coming to the table for people, especially in the Shopify ecosystem. So it's pretty fun to be in this ecosystem for myself. 

Sponsor: Rewind  

Nobody likes insurance, but everyone should have it. Mistakes do happen. Our partner Rewind can protect your Shopify store with automated backups of your most important data. 

Rewind should be the first app you install to protect your store against human error, misbehaving apps, or collaborators gone bad. It's like having your very own magic undo button. 

Trusted by over 70,000 businesses from side hustles to the biggest online retailers like Gymshark, Gatorade, and MVMT Watches

Best of all, respond to any of their welcome emails and mention Honest Ecommerce (this podcast) to get your first month absolutely free.

Chase Clymer  

So I do want to pivot the conversation just a little bit now because we touched on it a little bit before. But you are in-house at Kettle and Fire. But you are partnered with some external agencies. 

And let's talk about how you handle that relationship and set and level expectations for people out there that are just one step behind you. like What should they be under... How should they go into it? And what should they know?

Niccoló Gloazzo  

Yes. So we have an external agency that manages the development and design and then we have some junior developers in-house and designers as well. So it's pretty... It could be very challenging sometimes to manage an external agency, especially also when you are all remote. 

You're not next door so that you can easily communicate. And so talking about communication that's exactly what needs to be perfect. Communication, it's really, really important, especially while you're being remote, the agency's remote as well. 

So clearly communicate the desired outcome that you want. I think that's one of the key pillars [to success] is to have a strong communication and be efficient in working what you do. And then practice and good organization make it perfect over time. 

So being organized and [have a] clear communication toward a developer is the key. And then when they understand what you're looking for. 

So okay, I'm looking for building those specific split tests to one site for every 2 weeks with this concept that you need to provide design and then development. Okay, they get into their routine. And so then, also the outcome is always better and better and better.

Chase Clymer  

Yeah. I can agree with a lot of that stuff, especially when you get started with working with a partner. It could be a main agency, it could be a copywriter, it could be an email, it doesn't matter if you need to have a really solid kickoff call with them, or it can be called whatever. 

But that first interaction you should definitely almost lay down the law of like, "Hey, this is how we like to communicate. These are the tools that we like to use. And we're gonna add you to all these tools, because that's how our team works most efficiently." 

Now that we're doing this more and more partnering with other agencies and stuff, we're even talking about our tech stacks on these calls. We're like, "Hey, we're using this. Here's our GitHub. This is how everything works. And it's how we build things." 

On the design side, it was like, "We're all about Figma. We're gonna use figma together." And we have onboarding documents that explain the processes and workflows behind these things to just make the relationship that much better. 

Over-communicating at the beginning and setting expectations is going to go so far to just strengthen that relationship and increase the amount of awesome work that you can do together.

Niccoló Gloazzo  

Oh yeah. Exactly. And like you mentioned over communicating, I think it's good to over-communicate. I mean, not, you know, I don't do that often, but it's something I should improve on. 

But over-communicating... It's better to over-communicate than don't. And because then you'll miss out about some potential really key element that you were expecting. But because you didn't communicate, the agency didn't deliver.

Chase Clymer  

Yeah, it's a two way street, though. I'm obviously on the end of the agency. And we have clients that come on board and I always ask them. We're like, "Hey, how much time are you going to dedicate to this?" 

And that's a really eye opening question. Because some people don't think about it. They're like, "Oh, I thought you were gonna solve this problem for me?" We're like, "Well yeah, but I can't make all the decisions." 

There needs to be an open line of communication and you need to be able to respond to our questions or you're gonna delay things. We're gonna hit... We're gonna miss timelines. And there's... We can't work sometimes without input. 

Niccoló Gloazzo  

Yeah. Exactly. 

Chase Clymer  

Awesome. So what are some other things that you wanted to share with the audience that I forgot to ask you about today?

Niccoló Gloazzo  

Well, I think that one of the biggest lessons that I learned potentially while doing big sitewide tests... And here I talk, maybe, for merchants that actually have the possibility to do big site wide tests, is to always have secondary KPIs to track on. 

Because I remember once we did a really big test, and the main KPI was actually skyrocketing. We were trying to improve subscription opt-in rate and the subscription rate was going really, really well. 

So we said, "Whoa! Amazing! that's great!" But then we didn't really track some other secondary KPIs. 

So for example, the churn rate on day zero, and some other secondary metrics. And at the end, we had to roll back to the actual original control variation and don't implement a variation, because actually wasn't performing as good as we thought.

So that's, I think, one of the biggest lessons that I learned, which is before actually implementing the winning variation, make sure that all your secondary KPIs are also on track, and that you don't... You never overlook something that you were before things could work out.

Chase Clymer  

That's great advice right there. Everyone's always super impressed by the vanity metrics, which is usually like traffic or conversion rate, or even top-line revenue. But you gotta make sure that the secondary KPIs, which are more probably quality related…

Make sure that the traffic you're getting is the right traffic. Make sure that the people are actually becoming customers instead of just [buying] one and done. Yeah. There's a lot more to it. 

And the one thing about it is it's a little bit of art and a little bit of science to it. It's a pretty fun little area of Ecommerce that I'm always glad to have people on to educate me about it. I wouldn't say that I'm a CRO specialist by any means. I definitely understand user experience. And I always have probably great ideas, but it's fun to learn more about it.

Niccoló Gloazzo  

Oh, yeah, exactly. And then one last thing, I would say that. I think everyone should start testing as soon as possible. 

Even if you don't have the possibility to test like I mentioned before, you could even buy... You could even look at your low hanging fruit that maybe comes out from analytics or from... 

If you use Hotjar to track either specific qualitative feedback. Start testing and try to understand what's working and what's not working. So like I mentioned before, even with Unbounce, you could test two completely different landing page and understand, "Okay, what's the angle that works best for your type of audience?"

Chase Clymer  

Absolutely. So, as we round out the podcast here, you're doing a lot of really cool stuff. And also in your free time, I know that you're an educational partner. 

And you are launching a course in Italian. Also, plug your podcast as well, so everyone can kind of go check it out if they want to learn more about Shopify and Italian. (laughs)

Niccoló Gloazzo  

(laughs) Yes, for sure. 

Chase Clymer  

I don't know how many of my listeners are in Italy, but I can look it up. And it's always wild to me. 

That's the cool thing about this podcast is there are listeners all over the world. And it just surprises me that people actually care about the content that I put out. IThank you, everybody. I don't say that as enough as I should.

Niccoló Gloazzo  

(laughs) Yes. Well, you have one for sure. From Italy, which is me. And then others I don't know. But probably yes.

Chase Clymer  

Awesome. Well, what's what's the podcast so people can search for it?

Niccoló Gloazzo  

It's actually called Ecommerce Talks. So kind of easy Word that was the first one that they had in mind when I launched during lockdown back in March together with a course. So it shouldn't be there.

Chase Clymer  

Awesome. We'll make sure to put that in the show notes. Anything else before we go?

Niccoló Gloazzo  

Nope. That's that's it from my end.

Chase Clymer  

Awesome. Everyone, all the stuff we chatted about, we'll link to in the show notes as well. Thank you so much for coming on today and sharing all that awesome insight about CRO and AB testing. Nicco you have a great day.

Niccoló Gloazzo  

Thank you. Thanks. Thanks. You too.

Chase Clymer  

I cannot thank our guests enough for coming on the show and sharing their journey and knowledge with us today. We've got a lot to think about and potentially add to our businesses. Links and more information will be available in the show notes as well. 

If anything in this podcast resonated with you and your business, feel free to reach out and learn more at electriceye.io/connect. Also, make sure you subscribe and leave an amazing review. Thank you!