Moz Crawl Test: WordPress sites with and without /feed and /trackback entires?
-
I have multiple WP websites and on some of the websites, on my Moz Crawl test, I see an entry for every blog post but also entries for /feed and /trackback for that single blog post. For example,
www...com/someArticle
www....com/someArticle/feed
www...com/someArticle/trackback
1. Can anyone explain why the Crawl test is picking up the /feed and /trackback items? Is it simply because they are 301 redirects to the original post (www...com/someArticle)?
2. What setting(s) in WordPress are making this information appear? Or is it just that the site(s) that have the /feed and /trackback are displaying "normal" behavior for a WP site with a lot of trackbacks and feed entires?
3. Should /fee and /trackback, as well as /author be blocked in robots.txt?
Thanks in advance for your advice and input!
-
I have the same issue but instead of it redirecting to the parent post its just going to a 404 page.
-
So I solved the problem (or at least figured where it was coming from). On this particular site, under the comments area, there is a link for "trackback url" and a link for "comments rss feed". Naturally these are ../trackback and ../blog so that's why the crawl is picking them up. They are 301 redirected to the "parent" page so that's why they are not a duplicate content issue. Thank to everyone for their help!
-
1. If you check the source code of your blog posts, there must be some sort of link to the feeds - possibly even in the header. I'm not 100% on how the Moz crawler operates (if it only spiders <a>anchor links or if it spiders referenced links in the header - pretty sure the latter) - but either way that's how they're finding it, through some sort of link on the page.</a>
<a>You could try running a crawl with Screaming Frog SEO Spider and see if it also picks up the feed URLs and Screaming Frog will show you where it found the links as well.
2. Good question. Your theme may be displaying links to these things somewhere - the best way to find out is to crawl with Screaming Frog and it will show you which pages link to your feed and trackback URLs. Then if you don't need them, you can go into the editor and remove them from the code.
3. I agree with Thomas here, I would not block them with robots.txt - rather I would see if you can fix them at the source and remove the links if they are not needed.
-Dan</a>
-
Thanks, I'll check it out!
-
Hi, you should never block feeds they're really pretty beneficial to your site. Take a look at this from Joost it will explain it much better than I can
http://yoast.com/example-robots-txt-wordpress/
All the best sincerely, Thomas
-
Thank you.
When you say "TrackBacks are from people posting either identical or similar content to WordPress.com", what do you mean? I thought trackbacks were notifications of links back when someone links to your content?
And why does the codex recommend blocking feeds and trackbacks in robots.txt?
Thanks again!
-
the TrackBacks are from people posting either identical or similar content to WordPress.com I would follow up with that. unless that person is you.
No do not block a feed with robots.txt and do not block the TrackBacks use automatics Digital millennium act takedown if somebody is stealing your content.
Sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is this owned by moz?
We continue to get hundreds of spam links per month, and just found a link from here. I do not believe this is owned by moz. https://www.wapz.net/ Maybe I'm wrong? Thanks.
Moz Pro | | plahpoy0 -
Is the MOZ Community Ranking broken?
I have been busy and away from MOZ for a while, but over the years I have been an active member of the community. I have over 2000 Moz points and usually if you haven't been on in the specified time period you become unranked, however this usually updates in 24 hours of logging back in. I have been active again since Sunday and I am still unranked yet I have gained several more points answering a couple of questions. My question is - have I missed something and the system has changed (not according to the top users table) or is the system broken?
Moz Pro | | Matt-Williamson0 -
Moz can't see my link from an external site
Hi, I'm hoping someone can shed some light on this. I've had a followed link created from http://www.dubaisharks.com/ pointing to my website - http://www.yallarugby.com but in the reports it doesn't show that there is a link coming fro this website. The link has been active for a few months now, but still no record of it. Any ideas?
Moz Pro | | SeoSheikh0 -
Not provided traffic in Moz
Hello, I was wondering, if anyone knew how moz handles non provided keywords? is it not displayed, or is it labeled as something else? I know in WMT, it still displays it. Thanks for your help.
Moz Pro | | PeterRota1 -
Crawl diagnostics up to date after Magento ecommerce site crawl?
Howdy Mozzers, I have a Magento ecommerce website and I was wondering if the data (errors/warnings) from the Crawl diagnostics are up to date. My Magento website has 2.439 errors, mainly 1.325 duplicate page content and 1.111 duplicate page title issues. I already implemented the Yoast meta data plugin that should fix these issues, however I still see there errors appearing in the crawl diagnostics, but when going to the mentioned URL in the crawl diagnostics for e.g.: http://domain.com/babyroom/productname.html?dir=desc&targetaudience=64&order=name and checking the source code and searching for 'canonical' I do see: http://domain.com/babyroom/productname.html" />. Even I checked the google serp for url: http://domain.com/babyroom/productname.html?dir=desc&targetaudience=64&order=name and I couldn't find the url indexed in Google. So it basically means the Yoast meta plugin actually worked. So what I was wondering is why I still see the error counted in the crawl diagnostics? My goal is to remove all the errors and bring it all to zero in the crawl diagnostics. And now I am still struggling with the "overly-dynamic URL" (1.025) and "too many on-page links" (9.000+) I want to measure whether I can bring the warnings down after implementing an AJAX-based layered navigation. But if it's not updating it here crawl diagnostics I have no idea how to measure the success of eliminating the warnings. Thanks for reading and hopefully you all can give me some feedback.
Moz Pro | | videomarketingboys0 -
Strange nothing site ranking
Hi There. If you check who ranks for "credit cards" there is a website https://www.woolworthsmoney.com.au/ that is in position #5 This is a highly competitive keyword, but OpenSiteExporer.org cannot give me any backlinks for it. it says "No Data Available for this URL" The same thing happens in Market Samurai - no data 1. What are these guys doing that the others are not? 2. How come OSE can't pull any data for it?
Moz Pro | | SearchProduct0 -
How long does the seomoz crawl take?
It's been doing it's thing for over 48 hours now and Ive got less than 350 pages... is this norma? It's NOT the first crawl.
Moz Pro | | borderbound0 -
Site metric tool
Is there a tool on here (or anywhere else) where you can upload a list of sites (say 500 sites) and be given things like mozrank on it?
Moz Pro | | thefresh0