Work with us

Grow Your Business Online With HANABI

Let's get to work. If you are thinking about launching or updating your e-commerce store and you think we'd be a good fit, then we would love to hear from you. Fill out our project form below to discuss how we can help your business and we'll get back to you in 24 hours.

Get your free consultation

What services you are interested in?*

Please select

Tell us about the project*

What is your budget?

Please select
Work
25 Jun 2021
Web Development

Shopify Stores Can Now Edit Their Robots.txt File

Shopify has released a new feature to enable stores to edit their robots.txt file, enabling greater control for a Shopify store's SEO.

On June 18, Tobi Lutke – Shopify CEO, announced on Twitter that Shopify stores are now able to edit their robots.txt file which gives store owners more control over how search engines crawl their site.

The default robots.txt file in Shopify

All Shopify stores include a default robots.txt file that is optimal for Search Engine Optimisation (SEO) and Shopify notes that this works for most stores, but you can now edit the file as described in the steps below to:

  • allow or disallow certain URLs from being crawled
  • add crawl-delay rules for certain crawlers
  • add extra sitemap URLs
  • block certain crawlers

The robots.txt file is located at the root directory of your Shopify store's primary domain name (this is also the case for the majority of websites). For example johns-apparel.com/robots.txt.

Following the steps described below to edit this file, you will find the following default code used by Shopify to generate the robots.txt file. This incorporates Shopify's default rules and by leaving this code in place, you will ensure that if Shopify updates their own rules - they will automatically apply to your robots.txt file.

# we use Shopify as our ecommerce platform {%- comment -%} # Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file. {% endcomment %} {% for group in robots.default_groups %} {{- group.user_agent -}} {% for rule in group.rules %} {{- rule -}} {% endfor %} {%- if group.sitemap != blank -%} {{ group.sitemap }} {%- endif -%} {% endfor %}

Editing the robots.txt file in Shopify

It is impor­tant to note that while Shopi­fy pro­vides the abil­i­ty to edit this file, Shopi­fy Sup­port can’t help you with edits to this file. Incor­rect use of this file could result in loss of traf­fic or your store no longer being indexed by search engines.
If you are unsure, Shopi­fy Part­ners like us can help you to edit this code.

Steps

The fol­low­ing steps cre­ate a robots.txt.liquid file (rather than a robots.txt file that sites might tra­di­tion­al­ly use).

  1. From your Shopi­fy admin, go to Online Store > Themes.
  2. Click Actions, and then click Edit Code.
  3. Click Add a new tem­plate, and then select robots.txt.
  4. Click Cre­ate tem­plate.
  5. Make the changes that you want to make to the default tem­plate. For more infor­ma­tion on liq­uid vari­ables and com­mon use cas­es, refer to Shopi­fy’s Devel­op­er page Cus­tomize robots.txt.liquid.
  6. Save changes to the robots.txt.liquid file in your pub­lished theme.

Version control

It is also impor­tant to note that if your Shopi­fy store theme is cur­rent­ly being ver­sion-con­trolled and deployed using Shopi­fy’s The­meK­it, com­mand line changes or GitHub, then you should create/​make the changes to your robots.txt.liquid file through these processes.

To find out more, read Shopi­fy’s Edit­ing robots.txt.liquid help article.

Related articles

18 Jul 2022

The 2022 Digital Marketing Glossary

With the wide range of digital marketing channels and mediums, it can be hard to keep up with all the buzzwords and terminology. To help you, we've put together the latest digital marketing glossary of some of the most commonly used industry terms.

14 Jun 2022

Key Takeaways from the Google Marketing Live 2022 Keynote

Google shared the latest updates to its ads and commerce products at the annual Google Marketing Live event.

30 Mar 2022

Our commitment to 1% for the Planet

Hanabi is proud to share that we are now a member of 1% for the Planet. This means that we're committed to giving 1% of all sales back to the Planet every year.

Sign up to our newsletter for the latest digital marketing and e-commerce news.

Get in touch
Services


© 2022 Hanabi. Registered in England and Wales. No. 11764836. VAT Number: GB399204759. PRIVACY POLICY ACCESSIBILITY