How I can check if Google and other search engines will properly cache a page (a dynamic one)?
-
My site is currently disallowing search engine bots with the help of robots.txt. These dynamic pages can be crawled using Screamingfrog since they are linked to a static category page which is also linked to the homepage.
Thanks in advance!
-
Great idea using Screaming Frog first then :-
Then use Google Web Master Tools > Crawl > Fetch as Google to see what Google sees
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why is Google.co.uk showing so many US retailers on search queries?
Over the last couple of years Google seems to be showing more and more irrelevant search results in the field I am based in (dietary supplements) with searches for products coming back with results which highlight US vendors. Given I am in google.co.uk and actually logged in as well, why is Google so far off with their results lately?
Algorithm Updates | | predatornutrition0 -
Reasons for a sharp decline in pages crawled
Hello! I have a site I've been tracking using Moz since July. The site is mainly stagnant with some on page content updates. Starting the first week of December, Moz crawler diagnostics showed that the number of pages crawled decreased from 300 to 100 in a week. So did the number of errors through. So crawler issues went from 275 to 50 and total pages crawled went from 190 to 125 in a week and this number has stayed the same for the last 5 weeks. Are the drops a red flag? Or is it ok since errors decreased also? Has anyone else experienced this and found an issue? FYI: sitemap exists and is submitted via webmaster tools. GWT shows no crawler errors nor blocked URLs.
Algorithm Updates | | Symmetri0 -
Google Maps marker inconsistency
We just discovered that depending on the address format you enter into Google, you may come across incorrectly placed marker locations on Google Maps. Is this because our Google Places address format is not consistent with Google Maps' format? If so, when I go into Google Places to update the address format, am I going to have to go through the citation process all over again?
Algorithm Updates | | SSFCU0 -
How can I tell Google two sites are non-competing?
We have two sites, both English language. One is a .ca and the other is a .com, I am worried that they are hurting one another in the search results. I'd like to obviously direct google.ca towards the .ca domain and .com towards the .com domain and let Google know they are connected sites, non-competing.
Algorithm Updates | | absoauto0 -
How to get global search results on Google ? Also, is it possible to get results based on some other geographic location?
I don't want results based on my geographic location. When I am in India, I don't want local search results. In fact, I want results which are not dependent on my current location. Also, can I change my current location to some other city and will it affect the results ? For eg: While I am in London, can my search results be modified as if I am sitting in New York ?
Algorithm Updates | | EricMoore0 -
How Can I Prevent Duplicate Page Title Errors?
I am working on a website that has two different sections, one for consumers and one for business. However, the products and the product pages are essentially the same but, of course, the pricing and quantities may be different. We just have different paths based on the kind of customer. And, we get feeds from manufacturers for the content so it's difficult to change it. We want Google to index both sections of the site but we don't want to get hammered for duplicate page titles and content. Any suggestions? Thanks!
Algorithm Updates | | JillCS0 -
Changing the # of results per page in Google search settings displays totally different results. Why is this?
Curious what's going on here. This is the first time I've seen this before. What's happening is this ... In Google, I search for "mobile apps orange county" and get a standard list of 10 results. I go to Google's search settings in the top right corner of the page (button is grey with a gear) to change the number of results per page from 10 to 50 (also did 100). When I go back to Google and search again for "mobile apps orange county" I get a much larger list but with completely different results. This time around the top 10-12 are dominated by the same website (ocregister.com) What's going on here that Google would now show different results? Why is this one website all of a sudden dominating the first 12 results? Thanks everyone! ByteLaunch
Algorithm Updates | | ByteLaunch0 -
Google Update on the 6th July
Hi Mozzers, Has anyone noticed a Google update on the 6th July? A price comparison site I optimise has fallen off the SERPs for most generic terms, however still getting traffic for longer tail phrases. Cheers Aran
Algorithm Updates | | Entrusteddev0