12 Questions to Ask a Shopify Developer: UX & Beyond

There’s a long list of reasons why the design and development (UX/UI) team that built the first version of your site likely won’t be the team that takes your brand to $20 million, then $50 million, and on.

Maybe their skillset was best suited for your get-up-and-running early days, and not the strong and sustainable growth you need now.

Maybe you hired them mostly because they were within your budget and could get the job done (sorta), but they’re not a highly regarded team in their field.

Maybe you’ve been juggling two separate agencies for UX and UI, and you’d prefer the seamless integration of getting both needs met by one coordinated team. 

Maybe you went the inhouse designer and developer route and found that it’s just impossible to retain talent that grows and excels.

Best-in-class UX/UI is oxygen for eCommerce brands.

If you’re in search of results-driven UX/UI expertise from one house, there are some key factors to consider as you’re making your pick. 

Here’s a list of 12 questions to ask before you hire a UX/UI agency:

UX/UI Standards

1. What UX certifications has your team achieved?

Many people have become great designers and coders in their high school bedroom, so do certifications really matter? 

There are two UX certifications that really do matter. Baymard and NN/g. 

Here’s why you want a team that’s been approved and continues to be informed by these research-based organizations:

NN/g was founded by two of the most distinguished UX experts in the world, Don Norman and Jakob Nielson. They study real user behavior year after year, with large sample sizes to ensure the highest degree of accuracy. While NN/g focuses on general UX principles, they’ve placed a lot of emphasis on eCommerce over the last several years.  

Baymard is a web usability research institute based out of Copenhagen, Denmark. Their research methodologies are built on academic principles, and they translate those results in actionable articles, reports, and benchmark databases. The Baymard certification is one of the most difficult tests we’ve seen in the market – you’re only allowed 3 attempts to pass, and many don’t.

Every member of the Anatta UX team has to obtain both NN/g and Baymard certifications before they can be deployed to a project. That’s how much confidence we have in the validity of their research practices. 

* Important: When you’re interviewing an agency, confirm that the whole UX team has achieved all of these certifications, and not just the department head or a few seniors.

2. What factors do you consider when developing conversion rate optimization tests?

Listen extra carefully to how this question is answered. 

The typical metrics you’ll probably hear mentioned (conversion rates, AOV, click-through rates, bounce rate, or time on site) only provide one part of the picture. These are quantitative metrics – they tell you what’s working or not working. 

But quantitative data doesn’t tell you WHY a feature or design shift failed or was a success. To understand the ‘why’, you need qualitative metrics like customer interviews, post-purchase polls/surveys, and user testing.

You want an agency that actively prioritizes both sets of data when considering what to test, so you don’t waste money on a test that a rudimentary data analysis would have shown was pointless.

3. Where do you source your qualitative and quantitative data?

We mentioned two of the best above: Baymard and NN/g. They are truly the UX gold standard.

But there’s another we recommend. While they’re not a research firm per se, we’ve found CXL’s best practices summaries and tips to be in a class of their own. 

They focus on conversion optimization with data-driven marketing trainings, and offer some of the most in-depth marketing, analytics, and optimization content on the web.

Of course there are exceptions and outliers, but by and large, the data and analysis from these sources continues to be irrefutable, no matter what industry or product category you’re in. Their findings are based on real users, real behavior, and verifiable metrics. 

If an eCommerce agency isn’t constantly referring to research-based data sources, proceed with caution. 

4. What type of UX framework do you use to go from Ideation to Completed Feature?

The first question to ask here is whether they even have a UX framework at all (many agencies don’t). If not, you should probably stop right there. 

If they do, ask for details because there are a lot of frameworks out there – Lean UX methodology, for example. It’s important for you to know which framework they use and even more importantly, ask for specifics on how it was used on a previous project.

Here’s another detail we highly recommend: 

Find out if their designers start with a sketchpad. Yup, old school pen and paper. Starting with rough paper sketches helps to eliminate wasted hours on multiple design iterations. Only after getting your approval of a paper sketch should designers begin investing the time required to build LoFis and HiFis. 

Lastly, the framework must include a style library or component library. This makes creating HiFis way faster and also speeds up development by over 50%. 

5. What type of UI Development framework do you use to build and style features?

All Shopify stores are not built equally. You can tell a lot about a store’s theme by noticing how fast it performs, how quickly features get built, and how seamlessly pages work together. 

The HTML, CSS, and JS frameworks matter, so you need to confirm 1) that there is a  framework being used and 2) what the benefits and drawbacks are of that framework.

In the Shopify space, Slate and Timber are commonly used, but both are being retired, so you need an agency that either uses these frameworks with Shopify support, or has customized their own and can prove its effectiveness.  

Clear Communication

6. How many other clients will my designers and developers be working on?

Our clients have told us stories about rarely seeing their UX team, or only being “allowed” rushed conversations with developers. 

If the designers and developers assigned to your brand are also working on clients other than you, what does that mean for the amount of time and energy you’ll get? Ask for specifics on this.

7. How will my team communicate with the agency team members assigned to us?

  • Will you be added to their internal project management software so that you can see work in progress in real-time?
  • Will you meet with your entire agency team weekly vs bi-weekly vs monthly? And how long will those meetings be, typically?
  • Will you have a slack channel for quick communication?

8. Are there scheduled check-ins and deliverable reviews with directors of departments?

If senior leadership is out of reach, that’s a problem. These are the people who guide strategy and manage the workflow of your project on a daily basis. 

Instead of settling for short “How’s it going?” catch ups on the odd call, we highly recommend selecting an agency that schedules at least monthly meetings with all department heads. You should also have quick email and Slack access to these people whenever it’s needed.

9. What happens in an emergency? Who comes to the rescue?

While the eCommerce world isn’t brain surgery, urgent things do come up. Your site can’t underperform in any way when you’re driving tens of thousands of dollars in ad traffic each day, or you’re in the middle of a huge influencer campaign.

You should be clear on exactly who is your 24/7 contact person, and how to reach them to get time-sensitive concerns addressed immediately. 

10. How do you charge? Hourly or monthly?

Hourly means they’re incentivized by how many hours they put in. Monthly means it’s a flat amount so hours don’t matter. 

If you go the monthly route, check references to know what kind of approximate output you can expect per month. 

11. Is QA included?

You might be surprised by how many agencies either don’t provide QA, provide it but then tack it onto your next invoice, or expect you to do it yourself.

In addition, most do not do regression testing or functional testing per deployment, both of which are vital.

If you are in conversation with an agency that does include QA, confirm whether other designers and developers are doing the QA or if they have a department solely for this purpose. The meticulousness of dedicated QA experts is hard to match.

12. Are there any copywriters or narrative directors?

The upside of working with a UX/UI agency that also provides narrative direction or an actual copywriter, is that design and copy can be created collaboratively within one team. Instead of narrative and design operating in silos, team members are able to test and pivot quickly, which invariably leads to better results for you.

This also frees up your own copywriter to focus on social content, email flows, PR campaigns, etc.

The partnership between a brand’s internal team and their UX/UI agency is a crucial success component. 

We hope these 12 questions will help you find the right agency for you.

Subscription business? One last thing before you go…

If one of the perks you’re looking for with your next agency is a subscription solution that provides a completely branded experience, while developing loyalty and retention, we have a separate list of key questions to ask (and red flags to look out for).

Leave a Comment

Your email address will not be published.