Does Google crawl and index dynamic pages?
-
I've linked a category page(static) to my homepage and linked a product page (dynamic page) to the category page. I tried to crawl my website using my homepage URL with the help of Screamingfrog while using Google Bot 2.1 as the user agent. Based on the results, it can crawl the product page which is a dynamic.
Here's a sample product page which is a dynamic page(we're using product IDs instead of keyword-rich URLs for consistency):http://domain.com/AB1234567
Here's a sample category page:
Here's my full question, does the spider result (from Screamingfrog) means Google will properly crawl and index the property pages though they are dynamic?
-
Andy's correct. The answer is most likely yes. So if screaming frog find these pages, most likely the Google bot will as well. Also, as long as the spider can find the page, it doesn't matter if it's static or dynamic. However, If you have multiple parameters, Google may decide it's not a useful page and won't index it. Make sure to include a sitemap with all these url's as it may help in getting them indexed in case your site architecture is poor.
-
For the most part, Google can find their way well around dynamic pages with few problems. There are always going to be exceptions, but this tends to be when there are lots of components to the URL structure.
As long as it is pretty simple, you shouldn't have any problems.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I Report A SEO Agency to Google
Our competitor has employed the services of a spammy SEO agency that sends spammy links to our site. Though our rankings were affected we have taken the necessary steps. It is possible to send evidence to Google so that they can take down the site. I want to take this action so that other sites will not be affected by them again.
White Hat / Black Hat SEO | | Halmblogmusic0 -
Should I noindex shop page and blog page for SEO?
I have about 15 products in my store. Should I noindex shop and blog page for SEO? The reason I ask this question is because I see someone suggesting noindex archives pages. And the shop page is product archive and blog page is archive too, so should I choose index or noindex? Thanks!
White Hat / Black Hat SEO | | Helloiamgood0 -
How authentic is a dynamic footer from bots' perspective?
I have a very meta level question. Well, I was working on dynamic footer for the website: http://www.askme.com/, you can check the same in the footer. Now, if you refresh this page and check the content, you'll be able to see a different combination of the links in every section. I'm calling it a dynamic footer here, as the values are absolutely dynamic in this case. **Why are we doing this? **For every section in the footer, we have X number of links, but we can show only 25 links in each section. Here, the value of X can be greater than 25 as well (let's say X=50). So, I'm randomizing the list of entries I have for a section and then picking 25 elements from it i.e random 25 elements from the list of entries every time you're refreshing the page. Benefits from SEO perspective? This will help me exposing all the URLs to bots (in multiple crawls) and will add page freshness element as well. **What's the problem, if it is? **I'm wondering how bots will treat this as, at any time bot might see us showing different content to bots and something else to users. Will bot consider this as cloaking (a black hat technique)? Or, bots won't consider it as a black hat technique as I'm refreshing the data every single time, even if its bot who's hitting me consecutively twice to understand what I'm doing.
White Hat / Black Hat SEO | | _nitman0 -
How does google view...
I have two urls that are almost the same for example: www.mysite.co.uk/motoring/car_fuel www.mysite.co.uk/motoring/car-fuel both pages are very different, but on the same topic. How does google view the use of _ and - in urls? Will it see my urls as different? Please advise if you know the answer. Thank You.
White Hat / Black Hat SEO | | JamesT0 -
Googlebot stopped crawling
Hi All, One of my website stopped showing in SERP, after analysing in webmaster, found that Googlebot is not able to crawl. However it was working alright few days back. Try to investigate for panelisation, but no intimation found. I checked robot.txt for no follow etc but all seems to be ok. I resubmitted Sitemap in webmaster again, it crawled 250 pages out of 500 but it still site is not available in SERP (google), in bing it is ok. Pl suggest the best possible solutions to try. Thx
White Hat / Black Hat SEO | | 1akal0 -
Why won't my home page rank for branded terms?
Hello, I've been trying to figure out what factors are causing my home page not to rank for my branded terms. The site is www.lipozene.com and after the late April Google algorithm our rankings have disappeared off the map for the term "lipozene". Different element of the site show up in organic rankings, including our shopping cart (http://shop.lipozene.com) as high as page two. However, the home page is not ranking organically. On Yahoo & Bing we have never dropped out of the number 1 spot. We did engage in some link building activities, however we've removed nearly all of the links that were created by our SEO guy. I did NOT receive any notifications from Google regarding their link policy. If you search for lipozene.com we rank #1. Any thoughts on what we're missing thats causing us to not rank is greatly apprecaited. Thanks!
White Hat / Black Hat SEO | | lipoweb0 -
How is this obvious black hat technique working in Google?
Get ready to have your minds blown. Try a search in Google for any of these: proform tour de france tour de france trainer tour de france exercise bike proform tour de france bike In each instance you will notice that Proform.com, the maker of the bike, is not #1. In fact, the same guy is #1 every time, and this is the URL: www.indoorcycleinstructor.com/tour-de-france-indoor-cycling-bike Here's the fun part. Click on that result and guess where you go? Yup, Proform.com. The exact same page ranking right behind it in fact. Actually, this URL first redirects to an affiliate link and that affiliate link redirects to Proform.com. I want to know two things. First, how on earth did they do this? They got to #1 ahead of Proform's own page. How was it done? But the second question is, how have they not been caught? Are they cloaking? How does Google rank a double 301 redirect in the top spot whose end destination is the #2 result? PS- I have a site in this industry and this is how I caught it and why it is of particular interest. Just can't figure out how it was done or why they have not been caught. Not because I plan to copy them, but because I plan to report them to Google but want to have some ammo.
White Hat / Black Hat SEO | | DanDeceuster0 -
Has anyone seen this kind of google cache spam before?
Has anyone seen this kind of 'hack'? When looking at a site recently I found the Google cache version (from 28 Oct) strewn with mentions of all sorts of dodgy looking pharma products but the site itself looked fine. The site itself is www.istc.org.uk Looking in the source of the pages you can see the home pages contains: Browsing as googlebot showed me an empty page (though msnbot etc. returned a 'normal' non-pharma page). As a mildly amusing aside - when I tried to tell the istc about this, the person answering the phone clearly didn't believe me and couldn't get me off the line fast enough! Needless to say they haven't fixed it a week after being told.
White Hat / Black Hat SEO | | JaspalX0