CMS Auto Generated Sitemap Work Around?
-
Hey Moz Community,
The Shopify ecommerce platform auto generates xml sitemaps and robots.txt for you.
Frustratingly there is no way to augment either of these. If I noindex on a page it will still show up in the site map... Causing inconstancy with the sitemap submitted to GWT.
In theory if put the MY version of the sitemap on site and point GWT to MY version.. Would this solve the inconstancy ? Or would Googlebot go in and still crawl the default /sitemap.xml anyway?
Any suggestions and insight is greatly appreciated!
-
Hi Dylan,
I haven't worked much with the technical side of Shopify, so wasn't aware of this. Very prohibitive though.
I hope you can get this sorted OK.
-Andy
-
Hey Andy,
I haven't directly asked the powers that be at Shopify to turn it off.. However our developer here gave me a pretty technical explanation that essentially comes down to because Shopify runs on rails and the way it handles hitting the server changing something would change it for everyone.
Thanks for pointing out the robots file referencing the old sitemap by the way. Close one!
-
Hi Dylan,
When you upload a sitemap, you get the option to state which one, and the location. If you were to upload your own and then point to it, this should be fine. Just remember to either remove the auto-generated one and never add it.
Are you able to add to the robots file? If so, you can point to your site map in there too. However, if this is already in, don't add your own otherwise Google will see two and inconsistencies will occur.
Are they unwilling to turn this off for you?
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google only crawling a small percentage of the sitemap
Hi, The company which I work for have developed a new website for a customer, there URL is https://www.wideformatsolutions.co.uk I've created a sitemap which has 25,555 URL's. I submitted this to Google around 4 weeks ago and the most crawls that have ever occurred has been 2,379. I've checked everything I can think of, including; Speed of website Canonical Links 404 errors Setting a preferred domain Duplicate content Robots Txt .htaccess Meta Tags I did read that Matt Cutts revealed in an interview with Eric Enge that the number of pages Google crawls is roughly proportional to your pagerank. But I'm sure it should crawl more than 2000 pages. The website is based on Opencart, if anyone has experienced anything like this I would love hear from you.
Technical SEO | | chrissmithps0 -
Looking at creating some auto-generated pages - duplicate content?
Hi Everyone! We just launched a new version of our research site and the main CTA on the page sends users to a subdomain that's blocked by robots.txt. The subdomain link is one of our PPC landing pages and they would be duplicate content for every model (cars). We're also looking at a new content stream of deals pages, on the main domain. The thought process was that we could rank these pages for things like "Volkswagen golf deals" and also use them as canonical URLs from the PPC pages so that Panda doesn't get mad at us for sending hundreds of links to a subdomain that's blocked. It's going to take us a lot of time to write the copy for the deals pages, so if we auto-generate it by pulling a paragraph of copy from the car review, and numerical stats about that model, will it be classes as duplicate and/or is there any downside to doing it? Review Page: http://www.carwow.co.uk/car-reviews/Ford/Fiesta Deals Page: http://www.carwow.co.uk/deals/Ford/Fiesta PPC Landing Page: http://quotes.carwow.co.uk/buy/Ford/Fiesta I can't help but feel that this may all be a bit overkill and perhaps it makes more sense to build 1 central deals page per model with unique content that we can also send the PPC traffic to, then life any block from the quotes. subdomain. But that will take time and we'd also like a quick solution. I'd also question if it's even an issue to link to a blocked subdomain, Google adds the quote URL into the index but can't crawl it, which I've been told is bad - but is it bad enough to do something about? Thanks, JP
Technical SEO | | Matt.Carwow0 -
Delete or re-submit sitemaps for new products? How often?
When I add new products (approx. 10 a month), I usually delete the old sitemap and submit a new one. Is this ok to do, or should I just re-submit it with the new info included? Also, is once a month too much?
Technical SEO | | tiffany11030 -
Search result pages - noindex but auto follow?
Hi guys, I don't index my search pages, and currently my pages are tagged name="robots" content="noindex"> Do I need to specify follow or will it automatically be done? Thanks Cyto
Technical SEO | | Bio-RadAbs0 -
Is there an easy solution for duplicate page content on a drupal CMS?
I have a drupal 7 site www.australiacounselling.com.au that has over 5000 crawl errors (!). The main problem - close to 3000 errors- is I have duplicate page content. When I create a page I can create a URL alias for the page that is SEO friendly, however every time I do this, it is registering there are 2 pages with the same content. Is there a module that you're aware of that I can have installed that would allow me to show what is the canonical page? My developers seemed stumped and have given up trying to find a solution, but I'm not convinced that it should be that hard. Any ideas from those familiar with drupal 7 would be greatly appreciated!
Technical SEO | | ClintonP0 -
Why Do Transparent Networks Still Work
Hi Mozzers, My client has a major competitor that dominates several industry head terms. A check of their link profile reveals that they have 50 low DA domains that are identical to the main site, the only difference being that they all link to the main domain for these terms. They're not even attempting to disguise the network but it works. Can anyone tell me why? See: www.omega.com/vhpc/
Technical SEO | | waynekolenchuk0 -
Duplicate Video Onsite - How do you treat this in Sitemap?
How would you handle multiple pages using the same video content? As sometimes it does not make sense to have new videos for every product so you re purpose. Will you still get the effects in search results if the thumbs and video location is duplicated for some product urls?
Technical SEO | | andrewv0 -
Could somebody suggest a GOOD Wordpress XML sitemap generator?
We have been putzing around with Google XML Sitemaps Generator (a plug-in on Wordpress) for our Wordpress blog and we cannot get it to write an XML sitemap! Could somebody suggest a viable alternative that actually works? Thank you for your help! Jay
Technical SEO | | theideapeople0