Thanks for your help Travis, that was a really solid answer.

Hi, I'm not too sure what to do about this or what to think of it.

This magically appeared in my companies robots.txt file (literally magically appeared/text is below)

User-agent: Baiduspider
User-agent: Baiduspider-video
User-agent: Baiduspider-image
Disallow: /

I know that Baidu is the Google of China, but I'm not sure why this would appear in our robots.txt all of a sudden. Should I be worried about a hack? Also, would I want to disallow Baidu from crawling my companies website?

Thanks for your help,
-Reed

Hi Jeff,

Exactly what I was looking for. I don't think we'd need to change the way titles are displayed, just wanted to be sure it wouldn't count it as duplicate or see it as a keyword hiding technique.

Thanks for your help

Hi Bruce,

Thanks for your response, really helpful.

Cheers.

Hi Moz Community,

I know that hiding h1 tags isn't a good practice for SEO and google, but we have banners that look much nicer than the stock text Magento uses for its titles.

The banners have the same text and the h1 is in the source code, just not visible on front end. The option Magento gives is "hide title on the page." So I'm not sure if this is actually the bad way to hide it or if it's fine for search engines.

Thanks,
-Reed

Hey Doug, had another question for you. A big majority (90% of 18,000+ errors) of our 404 errors are coming from .jsp files from our old website.

Of course, it's not ideal to manually update or redirect these, but possibly write a script to automatically change them. Would it be beneficial to add this .jsp to our robots.txt file?

Thanks Doug, really helpful answer.

I am getting thousands of 404's but when I dive into them the majority of the 404 URLs can't be found in any of the "linked from" examples GWT gives me.

I think 301 redirects are the best option like you said and/or having a good 404 page.

Thanks,
-Reed

Hey Community,

Got a question about the disavow tool for you. My site is getting thousands of 404 errors from old blog/coupon/you name it sites linking to our old URL structure (which used underscores and ended in .jsp).

It seems like the webmasters of these sites aren't answering back or haven't updated their sites in ages so it's returning 404 errors. If I disavow these domains and/or links will it clear out these 404 errors in Google? I read the GWT help page on it, but it didn't seem to answer this question.

Feel free to ask any questions that may help you understand the issue more.

Thanks for your help,
-Reed

Thanks Gianluca, very helpful.

Thanks Andy,

Would it be best to have all the img alt tags set up before adding site-map?