How I can check if Google and other search engines will properly cache a page (a dynamic one)?
-
My site is currently disallowing search engine bots with the help of robots.txt. These dynamic pages can be crawled using Screamingfrog since they are linked to a static category page which is also linked to the homepage.
Thanks in advance!
-
Great idea using Screaming Frog first then :-
Then use Google Web Master Tools > Crawl > Fetch as Google to see what Google sees
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
People also searched for
Hi There, I am promoting an online events website and would like to know adding long tail keywords (4-6 words unique queries) under "People also searched for:" is a good or bad idea on event pages? My intention is not to stuff page with keywords, although I want to let search engine bot know that particular page is relevant for other mentioned queries as well. So Intent is to sent relevancy signal (via content). So the heading "People also searched for" could be changed to something like: Related searches Searches related to Related queries etc. Thanks
Algorithm Updates | | jamesinhere
James0 -
AngularJS - How to check?
Hi all, I have read and understood how to make sure that a site in angularJS can be crawled and indexed by search engines efficiently. However, I have a question around the pages and their meta data (title, description etc.). Since, headless browsers will run the code and then return the completed HTML pages, for an SEO, how it is possible to do a check on the meta data? This information wont be available in the code at first instance that SEOs can pull. How does an SEO get the meta data information from the code of the website? Also, how will tools like ScreamingFrog, Google Page Speed tool, Xenu etc. read websites developed in AngularJS?
Algorithm Updates | | Malika10 -
Google domain search
Hello all, I'm a newbie to SEO, so you'll have to bear with me. I just started a website LangleyHomeSaerch.com a few months ago and am having trouble ranking with google. When I search "Langley Home Search" with Yahoo or Bing, it comes up on the first page. However when I search it with google it doesn't seem to rank even in the first few hundred pages. The only way I can get a match from google is if I search "Langley HomeSearch" or "LangleyHomeSearch". I know due to google's newer algorithms that there is less importance put on domain name matches, but is this normal, or is there anything I can do to improve it? Thx, Colby Langley, BC
Algorithm Updates | | colbygedak0 -
With Google's Location Based Searches, Should I Include a City Name with My Keywords?
What I mean is when you search on Google it seems to pull information by your location so would it be helpful including the city name + keyword still for SEO or would it be just as helpful using just the keyword? For example, a client is in Alexandria, VA and has a computer repair shop so would "Alexandria computer repair" be as good or better than "computer repair"? Just a little curious. Thanks!
Algorithm Updates | | CodyOelker-AMICreativeStudio2 -
New Search Engine.... Vanoogle.com
I'd like to see google start a new search engine. They might call it Vanoogle.com (Vanilla Google). This search engine would not be stinked up with social data, freshness inclusions, crap from my last query, skewed based upon my IP, warped because of my browser, targeted because of my cookies, no personalization, no image results, product results, none of that stuff. Ads are OK if labeled. I just want a plain vanilla search. Something that I know is "clean". Just like the good olde days. Millions of people will start using it right away. 🙂 Would you use Vanoogle.com?
Algorithm Updates | | EGOL2 -
Test contet/pages indexed by search engines
During the web development stages of our Joomla CMS website, we have managed to get our site indexed for totally irrelevant test pages mainly to do with Joomla and some other equally irrelevant test content. How damaging is this to our domain from an SEO prospective and is there something we can do about it? When we do a site:domain.com search we see hundreds of testpages with test/irrelevant meta tags etc.
Algorithm Updates | | Fuad_YK0 -
Related Searches in Google
Hello, We're helping a client remove/minimize some negative information about their brand in Google's search results. Just curious about your take on if the related searches that appear at the bottom of Google search results can in any way be influenced or if it is more a combination of so many factors that any one person or organization wouldn't be able to change very easily? I've heard the related results could be influenced if enough queries generated overtake the "negative" queries done initially but I feel like that is venturing into black hat land a bit. thanks -Mike
Algorithm Updates | | mattmainpath0 -
What are the differences between Google SEO and Bing SEO?
I came across this question on why the poster's rankings in Bing/Yahoo were so much lower than his rankings in Google. One of the links responded with was a presentation Rand gave about the difference in ranking elements of Google and Bing. My purpose for looking into this is to boost rankings in Bing to be more in line with my Google rankings. My takeaways from Rand's presentation were that Bing likes shorter URLs than Google and it's better to have more links from more root domains with more precise anchor text. Unfortunately this presentation was given at last year's SMX Advanced and is almost a year old. Since then Microsoft has been accused of basically scraping the Google SERPs and Google unleashed at least two maybe three rabid Pandas. Needless to say the environment has changed. So my question is for those people who are happy with how they rank in Bing: What SEO factors are you seeing make a bigger impact in Bing vs. how they impact your Google rankings?
Algorithm Updates | | rball11