Moz Crawl Test: WordPress sites with and without /feed and /trackback entires?
-
I have multiple WP websites and on some of the websites, on my Moz Crawl test, I see an entry for every blog post but also entries for /feed and /trackback for that single blog post. For example,
www...com/someArticle
www....com/someArticle/feed
www...com/someArticle/trackback
1. Can anyone explain why the Crawl test is picking up the /feed and /trackback items? Is it simply because they are 301 redirects to the original post (www...com/someArticle)?
2. What setting(s) in WordPress are making this information appear? Or is it just that the site(s) that have the /feed and /trackback are displaying "normal" behavior for a WP site with a lot of trackbacks and feed entires?
3. Should /fee and /trackback, as well as /author be blocked in robots.txt?
Thanks in advance for your advice and input!
-
I have the same issue but instead of it redirecting to the parent post its just going to a 404 page.
-
So I solved the problem (or at least figured where it was coming from). On this particular site, under the comments area, there is a link for "trackback url" and a link for "comments rss feed". Naturally these are ../trackback and ../blog so that's why the crawl is picking them up. They are 301 redirected to the "parent" page so that's why they are not a duplicate content issue. Thank to everyone for their help!
-
1. If you check the source code of your blog posts, there must be some sort of link to the feeds - possibly even in the header. I'm not 100% on how the Moz crawler operates (if it only spiders <a>anchor links or if it spiders referenced links in the header - pretty sure the latter) - but either way that's how they're finding it, through some sort of link on the page.</a>
<a>You could try running a crawl with Screaming Frog SEO Spider and see if it also picks up the feed URLs and Screaming Frog will show you where it found the links as well.
2. Good question. Your theme may be displaying links to these things somewhere - the best way to find out is to crawl with Screaming Frog and it will show you which pages link to your feed and trackback URLs. Then if you don't need them, you can go into the editor and remove them from the code.
3. I agree with Thomas here, I would not block them with robots.txt - rather I would see if you can fix them at the source and remove the links if they are not needed.
-Dan</a>
-
Thanks, I'll check it out!
-
Hi, you should never block feeds they're really pretty beneficial to your site. Take a look at this from Joost it will explain it much better than I can
http://yoast.com/example-robots-txt-wordpress/
All the best sincerely, Thomas
-
Thank you.
When you say "TrackBacks are from people posting either identical or similar content to WordPress.com", what do you mean? I thought trackbacks were notifications of links back when someone links to your content?
And why does the codex recommend blocking feeds and trackbacks in robots.txt?
Thanks again!
-
the TrackBacks are from people posting either identical or similar content to WordPress.com I would follow up with that. unless that person is you.
No do not block a feed with robots.txt and do not block the TrackBacks use automatics Digital millennium act takedown if somebody is stealing your content.
Sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why my site not crawl?
hi all me allow rogerbot in robots.txt but rogerbot can't crawl my site my site: toska-co.ir
Moz Pro | | jahanidawodi0 -
What exactly is Moz Analytics?
I've taken a look at this page - http://moz.com/products/tour and I'm trying to determine if Moz Analytics is live yet? I don't think it is, just wanted to make sure. Because right now I'm just seeing all the same tools as we've always had + Fresh Web Explorer. Anyways, let me know if I'm missing something.
Moz Pro | | CAndrew14.1 -
Rogerbot did not crawl my site ! What might be the problem?
When I saw the new crawl for my site I wondered why there are no errors, no warning and 0 notices anymore. Then I saw that only 1 page was crawled. There are no Error Messages or webmasters Tools also did not report anything about crawling problems. What might be the problem? thanks for any tips!
Moz Pro | | inlinear
Holger rogerbot-did-not-crawl.PNG0 -
I am trying to find inbound links for one of my site urls. My question is does SEOMoz able to track all internal links as the Open Site Explorer shows 0 internal links?
It shows 0 internal links when I am pretty sure we have multiple internal links.Should we use absolute urls or relative urls for internal links?
Moz Pro | | SulekhaUSLLC0 -
Crawl Diagnostic | Starter Crawl taken 14hrs.. so far
We started a starter crawl 14hrs ago and it's still going, can anyone help on why this is taking so long, when it says '2 hrs' on the interface.. Thanks, Rory
Moz Pro | | RoryMacDonald0 -
SEO MOZ Timezone
I have looked & looked but I can not figure out how to set MY timezone on SEOMOZ? The dates are all wrong in my reports ... **Organic traffic data ** Last Data Update: 04/16/2012 Crawl Diagnostics Last Crawl Completed: Apr. 20th, 2012 Next Crawl Starts: Apr. 25th, 2012 Keyword Rankings "Your keywords are updated weekly on Saturday. The last update was April 19th, 2012." Yet today is Friday the 20th 2012??? It seems like each segment of the information is updated on a differing date?
Moz Pro | | Mcarle0 -
How to Link build for our site
Hello, A client of mine has a site that gives advice to people. It's moderately competitive. It's a content site. Using Open Site explorer, I looked up competitor's top pages and backlinks, and there were things such as sites that were heavy on directories, sites heavily on editorial mension, sites on juicy topics that I don't want to cover, and the rest was pretty much junk. Just so that you know, I only link build with quality websites that look strong, long term, have a nice design, are high quality, and are not a directory (with a few exceptions) What's the standard way to link build with a site like this? We've got 7 articles of good content that is from me spending hours learning about the subject and using my skills as a life coach. Right now I'm just adding quality content and social media. I am not a top expert in the field, but I'm learning as much as possible. How do we link build?
Moz Pro | | BobGW0 -
How long does a crawl take?
A crawl of my site started on the 8th July & is still going on - is there something wrong???
Moz Pro | | Brian_Worger1