CMS Auto Generated Sitemap Work Around?
-
Hey Moz Community,
The Shopify ecommerce platform auto generates xml sitemaps and robots.txt for you.
Frustratingly there is no way to augment either of these. If I noindex on a page it will still show up in the site map... Causing inconstancy with the sitemap submitted to GWT.
In theory if put the MY version of the sitemap on site and point GWT to MY version.. Would this solve the inconstancy ? Or would Googlebot go in and still crawl the default /sitemap.xml anyway?
Any suggestions and insight is greatly appreciated!
-
Hi Dylan,
I haven't worked much with the technical side of Shopify, so wasn't aware of this. Very prohibitive though.
I hope you can get this sorted OK.
-Andy
-
Hey Andy,
I haven't directly asked the powers that be at Shopify to turn it off.. However our developer here gave me a pretty technical explanation that essentially comes down to because Shopify runs on rails and the way it handles hitting the server changing something would change it for everyone.
Thanks for pointing out the robots file referencing the old sitemap by the way. Close one!
-
Hi Dylan,
When you upload a sitemap, you get the option to state which one, and the location. If you were to upload your own and then point to it, this should be fine. Just remember to either remove the auto-generated one and never add it.
Are you able to add to the robots file? If so, you can point to your site map in there too. However, if this is already in, don't add your own otherwise Google will see two and inconsistencies will occur.
Are they unwilling to turn this off for you?
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Snippet not work for my article in google
I add the script for star snippet in my website but not work in my posts you can see it in this URL https://youtech.ooo/showbox-apk-download/ when I searched in google my custom keyword "Showbox" my competitor show with star snippet in SERP but my site doesn't show snippet stars. Thank You!
Technical SEO | | JackJasonn1 -
"Url blocked by robots.txt." on my Video Sitemap
I'm getting a warning about "Url blocked by robots.txt." on my video sitemap - but just for youtube videos? Has anyone else encountered this issue, and how did you fix it if so?! Thanks, J
Technical SEO | | Critical_Mass0 -
Which Sitemap to keep - Http or https (or both)
Hi, Just finished upgrading my site to the ssl version (like so many other webmasters now that it may be a ranking factor). FIxed all links, CDN links are now secure, etc and 301 Redirected all pages from http to https. Changed property in Google Analytics from http to https and added https version in Webmaster Tools. So far, so good. Now the question is should I add the https version of the sitemap in the new HTTPS site in webmasters or retain the existing http one? Ideally switching over completely to https version by adding a new sitemap would make more sense as the http version of the sitemap would anyways now be re-directed to HTTPS. But the last thing i can is to get penalized for duplicate content. Could you please suggest as I am still a rookie in this department. If I should add the https sitemap version in the new site, should i delete the old http one or no harm retaining it.
Technical SEO | | ashishb010 -
CMS on autopilot is happily creating duplicate pages - advice?
Hi, our ecommerce CMS (Magento) is creating a bunch of pages with very little content and no user value like this: http://goo.gl/UU2vl This particular example is the by product of a product filtering page, which has the format www.mywebsite/explore/index/loaddata/id/10/. These pages have no content other than images - also the pages don't have page titles and are therefore being flagged in webmaster tools as requiring HTML improvements We also have CMS auto generated pages like this: www.mysite.comhttp/review/product/list/id/7 where the page is effectively a duplicate of the product page, and this is giving us pages being flagged by webmastertools as having duplicate title tags. Should we exclude these two type of page via robots.txt or take another approach, like not worry about them 🙂 many thanks, any help gratefully received.
Technical SEO | | w1ll1am0 -
The search button on my site no longer works. Any fixes?
I have a wordpress site (can be found at theFlooringGirl.com). On all the pages, there is a search button so you can search for words on the site, but it no longer seems to work. My original webdesigner put it in, and I can't seem where to find it/where to edit/fix it. I finally found something that mentions "search" in the widget section and it is in the tool bar, but I can't find where/how to edit/fix it. BTW, I do have two other smaller sites that have this button and it still works there...so if I could find where to edit, I could theoretically copy and paste the code...if I knew where). Any advice? Thx.
Technical SEO | | Jborgueta0 -
Auto genrated content problem?
Hi all, I operate a Dutch website (sneeuwsporter.nl), the website is a a database of European ski resorts and accommodations (hotels, chalets etc). We launched about a month ago with a database of about 1700+ accommodations. Of every accommodation we collected general information like what village it is in, how far it is from the city centre and how many stars it has. This information is shown in a list on the right of each page (e.g. http://www.sneeuwsporter.nl/oostenrijk/zillertal-3000/mayrhofen/appartementen-meckyheim/). In addition a text of this accomodation is auto generated based on some of the properties that are also in the list (like distance, stars etc). Below the paragraph about the accommodation is a paragraph about the village the accommodation is located in, this is a general text that is the same with all the accommodations in this village. Below that is a general text about the resort area, this text is also identical on all the accommodation pages in the area. So a lot of these texts about the village and area are used many times on different pages. Things went well at first and every day we got more Google traffic, and more and more pages. But a few days ago our organic traffic took a near 100% dive, we are hardly listed anymore and if we are at very low places. We expect the Google gave us a penalty. We expect this to be the case because of 2 reasons: we have auto generated text that only vary slightly per page we re-use the content about villages and area's on many pages We quickly removed the content of the villages and resort area's because we are pretty sure that this is definitely something Google does not want. We are less sure about the auto generated content, is this something we should remove as well? These are normal readable text, they just happen to be structured more or less the same way on every page. Finally, when we made these and maybe some other fixes, what is the best and quickest ways to let Google see us again and show them we improved? Thanks in advance!
Technical SEO | | sneeuwsporter0 -
Safe way to auto geo redirect
Hi, looking for a second opinion on this please.... I own a couple of web stores, one targets UK and the other USA (they are both the same store more or less just different products targeted at different location). The uk runs on a .co.uk domain and the US on a .us domain. Is there a safe way that I could auto redirect search engine traffic to the right location? Let's say the toys page of the .co.uk is ranking well in google uk and appears high in google us too, obviously I would want the USA users to visit the toys page for the US store rather than the UK one. Ideally I would employ a geo redirect script so if a USA user clicks on the UK domain they are redirected to the USA site but would Google frown on that? Hope that makes sense? Thanks
Technical SEO | | GrumpyCarl0 -
How does a sitemap affect the definition of canonical URLs?
We are having some difficulty generating a sitemap that includes our SEO-friendly URLs (the ones we want to set as canonical), and I was wondering if we might be able to simply use the non-SEO-friendly, non-canonical URLs that the sitemap generator has been producing and then use 301 redirects to send them to the canonical. Is there a reason why we should not be doing this? We don't want search engines to think that the sitemap URLs are more important than the pages to which they redirect. How important is it that the sitemap URLs match the canonical URLs? We would like to find a solution outside of the generation of the sitemap itself as we are locked into using a vendor’s product in order to generate the sitemap. Thanks!
Technical SEO | | emilyburns0
