Will Google Revisit a 403 Page
-
Hi,
We've got some pretty strict anti-scraping logic in our website, and it seems we accidentally snared a Googlebot with it.
About 100 URL requests were responded to with a 403 Forbidden error. The logic has since been updated, so this should not happen again.
I was just wondering if/when Googlebot will come back and try those URLs again. They are linked from other pages on the site, and they are also in our sitemap.
Thanks in advance for any assistance.
-
Thanks for the reply Alan. We'll wait for them to crawl the site again.
-
My belief is that they always visit a page a few times before they make a decision for this reason.
If you have links to the page than they will certainly revisit.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How will changing my website's page content affect SEO?
Our company is looking to update the content on our existing web pages and I am curious what the best way to roll out these changes are in order to maintain good SEO rankings for certain pages. The infrastructure of the site will not be modified except for maybe adding a couple new pages, but existing domains will stay the same. If the domains are staying the same does it really matter if I just updated 1 page every week or so, versus updating them all at once? Just looking for some insight into how freshening up the content on the back end pages could potentially hurt SEO rankings initially. Thanks!
Intermediate & Advanced SEO | | Bankable1 -
Highly interlinking a particular page shows up in Google search results?
Hi All, As we can see the below statement from Google about internal linking: "The number of internal links pointing to a page is a signal to search engines about the relative importance of that page."
Intermediate & Advanced SEO | | vtmoz
https://support.google.com/webmasters/answer/138752?hl=en So if we interlink a page highly than other pages, will it rank on search results instead of homepage? Moreover if the page have "keyword" in URL slug...like www.website.com/keyword. Thanks0 -
Google pulling brand snippets from only some of my pages. Different settings or are they just being selective?
Hi, Moz community For some of the category-pages, Google is showing some of the brands in the SERP, like this: http://www.screencast.com/t/62wldbwc
Intermediate & Advanced SEO | | Inevo
This is the page-url: https://www.gsport.no/sport/loep/lopeklaer/loepebukse For other category-pages that seemingly is built with similar code and settings, Google doesn't show brands in the snippet: http://www.screencast.com/t/zU9cg7odf
The page-url: https://www.gsport.no/sport/loep/lopeklaer/loepejakke This all begs the questions:
If the two pages contain the same code/html in terms of schema.org / rich snippets, why is Google choosing to display the brands in the SERP for only one of them? And is there something I can do in order to make them display the brands for all my pages? Thank you
Sigurd Bjurbeck, INEVO (digital agency)0 -
How does putting a trial sign up code mid-blog post effect SEO? Do you think it will make my content seem less pleasing, therefor decrease the page rank??
I'm working on driving trials for our product - we have a number of blog posts that rank on page #1 of Google, and we get 2-3 trial sign ups per day from them. I'd like to put trial signup boxes about midway down each post to see if I can increase the number of trial signups that come directly from our blog. Do you think I can be "penalized" for this, since it's mid- blog-post content? Do you think Google will view this negatively?
Intermediate & Advanced SEO | | Karibeaulieu0 -
Consistent Ranking Jumps Page 1 to Page 5 for months - help needed
Hi guys and gals, I have a really tricky client who I just can't seem to gain consistency with in their SERP results. The keywords are competitive but what the main issue I have is the big page jumps that happen pretty much on a weekly basis. We go up and down 40 positions and this behaviour has been going on for nearly 6 months.
Intermediate & Advanced SEO | | Jon_bangonline
I felt it would resolve itself in time but it has not. The website is a large ecommerce website. Their link profile is OK in regards to several high quality newspaper publication links, majority brand related anchor texts and the link building we have engaged in has all been very good i.e. content relevant / high quality places. See below for some potential causes I think could be the reason: The on page SEO is good however the way their ecommerce website is setup they have formed a substantial amount of duplicate title tags. So in my opinion this is a potential cause. The previous web developer set-up 301 redirects all to their home page for any 404 errors. I know best practice is to go to the most relevant pages, however could this be a potential issue? We had some server connectivity issues show up in webmasters tools but that was for 1 day about 4 months ago. Since then no issues. they have quite a few 'blocked URLs' in their robots.txt file, e.g. Disallow: /login, Disallow: /checkout/ but to me these seem normal and not a big issue. We have seen a decrease over the last 12 months in Webmasters Tools of 'total indexed web pages' from 5000 to 2000 which is quite an odd statistic. Summary So all in all I am a tad stumped. We have some duplicate content issues in title tags, perhaps not following best practice in the 301 redirects but other than that I dont see any major on page issues, unless I am missing something in the seriousness of what I have listed.
Finally we have also do a bit of a cull of poor quality links, requesting links to be removed and also submitting a 'disavow' of some really bad links. We do not have a manual penalty though. Thoughts, feedback or comments VERY welcome.0 -
Why the archive sub pages are still indexed by Google?
Why the archive sub pages are still indexed by Google? I am using the WordPress SEO by Yoast, and selected the needed option to get these pages no-index in order to avoid the duplicate content.
Intermediate & Advanced SEO | | MichaelNewman1 -
Google + under Google business domain email account
Hello there, I have a quick and straight question and I am hoping to find answer here. What do we do with a G+ profile that was set up through a business domain's email account that is used by more than one person? We want to use the company name, but we can't as it is considered personal email account although it is under business domain verified by Google. Is there a way that we ask Google to change it and allow us to use the name of the company or should we just deactivate it? Thanks in advance!
Intermediate & Advanced SEO | | montauto0 -
Google places page related places links to competitor
I noticed on a lot of Google places pages i create for my clients Google seems to put related places links at the bottom of the page which links directly to their competitors. how can i remove control or avoid these links been placed? Also any tips on improving the places page would be greatly appreciated thanks
Intermediate & Advanced SEO | | Bristolweb0