Site structure: Any issues with 404'd parent folders?

[dsbud]

Is there any issue with a 404'd parent folder in a URL? There's no links to the parent folder and a parent folder page never existed. For example say I have the following pages w/ content:

/famous-dogs/lassie/
/famous-dogs/snoopy/
/famous-dogs/scooby-doo/

But I never (and maybe never plan to) created a general **/famous-dogs/ **page. Sitemaps.xml does not link to it, nor does any page on my site.

Is there any concerns with doing this? Am I missing out on any sort of value that might pass to a parent folder?

[willcritchlow]

Yeah - there is various speculation about how signals or authority traverse folder structures (see for example this whiteboard Friday ) but I haven't seen anything suggesting it's permanent - all of this may be an argument for adding /famous-dogs/ at some point, but I wouldn't personally stress about it not being there at launch.

[willcritchlow]

Yeah. I'd just leave it as a 404 in that case

[dsbud]

In my scenario, considering I might add a parent "famous dogs" page at some point, it'd probably best to leave robots.txt alone, right?

[dsbud]

Thanks for the response. This is what I expected.

I swear I read somewhere that Google may pass some form of value from a child to a parent. i.e. "/famous-dogs/lassie/" could pass some value to "/famous-dogs/", absent any links. Can't find the source, but I suppose I'm a bit worried that I'd permanently lose out on some value if the parent does not exist initially. Considering I may add a "famous dogs" parent page at some point.

[willcritchlow]

PS - if you're worried about the crawling, you could always block it in robots.txt if you really wanted (but unless it's a huge site I wouldn't bother). Note - if you do go this route, do it carefully so as not to block all contents of the folder at the same time!

[willcritchlow]

The short answer is that there should be no harm going with your proposed approach.

Longer version: I believe there are cases where Google has tried to crawl a directory like "/famous-dogs/" in your example purely because it appears as a sub-folder in the paths of other pages even though there are not any direct links to it. But even if it does crawl it, if you don't have or intend to have a page there, a 404 is a perfectly valid response.

In general, while there could be a case that it's worth creating a "/famous-dogs/" page if there is search demand you can fulfil, until or unless you do, there is no harm in it returning a 404 response.

[dsbud]

Seems odd that indexers would care if a parent directory page exists or not. Is there any proof that Google will attempt crawl parent folder pages that aren't in sitemaps.xml and aren't linked to anywhere else?

Perhaps I'm slowly building out my site. Depending on the material/approach, it might make sense to release a page talking about a sub-category (lassie) before releasing content about a parent category (famous dogs). Or maybe "famous dogs" is such low search volume that it doesn't make sense to spend time creating a parent "famous dogs" page.

If I'm understanding correctly, with the above you're effectively telling me to:

1. Build a parent category page. If I don't plan on investing much time/effort into the parent page, noindex it.

2. Reorganize my site folder structure.

Neither seem like a great option.