Moz Crawl Test: WordPress sites with and without /feed and /trackback entires?

[Titan552]

I have multiple WP websites and on some of the websites, on my Moz Crawl test,  I see an entry for every blog post but also entries for /feed and /trackback for that single blog post.  For example,

www...com/someArticle

www....com/someArticle/feed

www...com/someArticle/trackback

1. Can anyone explain why the Crawl test is picking up the /feed and /trackback items?  Is it simply because they are 301 redirects to the original post (www...com/someArticle)?

2. What setting(s) in WordPress are making this information appear?  Or is it just that the site(s) that have the /feed and /trackback are displaying "normal" behavior for a WP site with a lot of trackbacks and feed entires?

3. Should /fee and /trackback, as well as /author be blocked in robots.txt?

Thanks in advance for your advice and input!

[jampaper]

I have the same issue but instead of it redirecting to the parent post its just going to a 404 page.

[Titan552]

So I solved the problem (or at least figured where it was coming from).  On this particular site, under the comments area, there is a link for "trackback url" and a link for "comments rss feed".  Naturally these are ../trackback and ../blog so that's why the crawl is picking them up.  They are 301 redirected to the "parent" page so that's why they are not a duplicate content issue.  Thank to everyone for their help!

[evolvingSEO]

1. If you check the source code of your blog posts, there must be some sort of link to the feeds - possibly even in the header. I'm not 100% on how the Moz crawler operates (if it only spiders <a>anchor links or if it spiders referenced links in the header - pretty sure the latter) - but either way that's how they're finding it, through some sort of link on the page.</a>

<a>You could try running a crawl with Screaming Frog SEO Spider and see if it also picks up the feed URLs and Screaming Frog will show you where it found the links as well.

2. Good question. Your theme may be displaying links to these things somewhere - the best way to find out is to crawl with Screaming Frog and it will show you which pages link to your feed  and trackback URLs. Then if you don't need them, you can go into the editor and remove them from the code.

3. I agree with Thomas here, I would not block them with robots.txt - rather I would see if you can fix them at the source and remove the links if they are not needed.

-Dan</a>

[Titan552]

Thanks, I'll check it out!

[BlueprintMarketing]

Hi, you should never block feeds they're really pretty beneficial to your site. Take a look at this from Joost it will explain it much better than I can

http://yoast.com/example-robots-txt-wordpress/

All the best sincerely, Thomas

[Titan552]

Thank you.

When you say "TrackBacks are from people posting either identical or similar content to WordPress.com", what do you mean?  I thought trackbacks were notifications of links back when someone links to your content?

And why does the codex recommend blocking feeds and trackbacks in robots.txt?

Thanks again!

[BlueprintMarketing]

the TrackBacks are from people posting either identical or similar content to WordPress.com I would follow up with that.  unless that person is you.

No do not block a feed with robots.txt and do not block the TrackBacks use automatics Digital millennium act takedown if somebody is stealing your content.

Sincerely,

Thomas