Sudden increase in number of indexed URLs. How ca I know what URLs these are?
-
We saw a spike in the total number of indexed URLs (17,000 to 165,000)--what would be the most efficient way to find out what the newly indexed URLs are?
-
You can also try searching your URL in google using exact paramaters...
inurl:http://www.yourdomain.com
or if you dont have a www.
inurl:http://yourdomain.com
-
The rest are most likely put into the supplemental index (are duplicates). I'd review GWT and analyze crawl stats, crawl errors, and html improvements.
Do you have 165k pages on your site? If not, it's probably some sort of error.
-
Hi Oleg,
Just tried that, but, it is only showing 300 URLs for the past week and 600 for the past month..
-
You can use the SEOmoz OSE you can use your google webmaster tools and you can also use Majestic . 17,000 to 165,000 indexed pages is a huge jump.
-
Search site:yourdomain.com, click on "Search Tools" in the left column and choose "Last week" (or within whatever period you saw the increase in indexed pages).
Or, you can choose "Past Year" and then choose "Sort by Date" - this will give you the latest links indexed in order.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Index Bloat: Canonicalize, Redirect or Delete URLs?
I was doing some simple on-page recommendations for a client and realized that they have a bit of a website bloat problem. They are an ecommerce shoe store and for one product, there could be 10+ URLs. For example, this is what ONE product looks like: example.com/products/shoename-color1 example.com/products/shoename-color2 example.com/collections/style/products/shoename-color1 example.com/collections/style/products/shoename-color2 example.com/collections/adifferentstyle/products/shoename-color1 example.com/collections/adifferentstyle/products/shoename-color2 example.com/collections/shop-latest-styles/products/shoename-color1 example.com/collections/shop-latest-styles/products/shoename-color2 example.com/collections/all/products/shoename-color1 example.com/collections/all/products/shoename-color2 ...and so on... all for the same shoe. They have about 20-30 shoes altogether, and some come in 4-5 colors. This has caused some major bloat on their site and I assume some confusion for the search engine. That said, I'm trying to figure out what the best way to tackle this is from an SEO perspective. Here's where I've gotten to so far: Is it better to canonicalize all URLs, referencing back to one "main" one, delete all bloat pages re-link everything to the main one(s), or 301 redirect the bloat URLs back to the "main" one(s)? Or is there another option that I haven't considered? Thanks!
Intermediate & Advanced SEO | | AJTSEO0 -
What to do with parameter urls?
We have a ton of ugly parameter urls that are coming up in google, in semrush, etc. What do we do with them? I know they can cause issues. EX https://www.hibbshomes.com/wp-content/themes/highstand/assets/js/cubeportfolio/js/jquery.cubeportfolio.min.js?ver=6.3
Intermediate & Advanced SEO | | stldanni0 -
Need help in de-indexing URL parameters in my website.
Hi, Need some help.
Intermediate & Advanced SEO | | ImranZafar
So this is my website _https://www.memeraki.com/ _
If you hover over any of the products, there's a quick view option..that opens up a popup window of that product
That popup is triggered by this URL. _https://www.memeraki.com/products/never-alone?view=quick _
In the URL you can see the parameters "view=quick" which is infact responsible for the pop-up. The problem is that the google and even your Moz crawler is picking up this URL as a separate webpage, hence, resulting in crawl issues, like missing tags.
I've already used the webmaster tools to block the "view" parameter URLs in my website from indexing but it's not fixing the issue
Can someone please provide some insights as to how I can fix this?0 -
Website Suddenly dropping rank
Morning Moz Fans: My URL is: http://goo.gl/Dhbjwj According to MOZ, which we are tracking this URL with, somewhere between the 3rd Feb and 10th the domain went from being fairly well indexed to being dropped to pages further back than 6-7, for pretty much everything, even the company name was only registering at the bottom of page one. Around this time we were transferring the website from .php into wordpress, so we were creating new pages, called by the same names and all the same content but we created the wordpress area in a sub domain of the website. Again around this time we had an issue with the blog area and had to take it down for 4-5 weeks due to some errors which meant google wouldn't have been able to crawl these pages properly, but the rest of the website was up and running. We also discovered recently that the company have and use this domain http://goo.gl/5JvDUH So my question is, what do you think caused the problem? has it been premaritally penalised? is there a way I can get google to specifically look at it and is there any more i can do?Thank you for your help
Intermediate & Advanced SEO | | popcreativeltd0 -
Why is this site not indexed by Google?
Hi all and thanks for your help in advance. I've been asked to take a look at a site, http://www.yourdairygold.ie as it currently does not appear for its brand name, Your Dairygold on Google Ireland even though it's been live for a few months now. I've checked all the usual issues such as robots.txt (doesn't have one) and the robots meta tag (doesn't have them). The even stranger thing is that the site does rank on Yahoo! and Bing. Google Webmaster Tools shows that Googlebot is crawling around 150 pages a day but the total number of pages indexed is zero. It does appear if you carry out a site: search on Google however. The site is very poorly optimised in terms of title tags, unnecessary redirects etc which I'm working on now but I wondered if you guys had any further insights. Thanks again for your help.
Intermediate & Advanced SEO | | iProspect-Ireland0 -
URL for offline use.
Hi there, We currently have a url www.example.com/health/back-pain/ We are wanting to promote this page on our product packaging however making the URL simpler www.example.com/back-pain/ is it just a case of using a 301? are there any issues here? Thanks for any feedback
Intermediate & Advanced SEO | | Paul780 -
Can Anybody Link to my URL to Hurt SEO? Weird URL pointing at my Domaine!
Our ranking has drop since a few weeks. I did not do any major change in my site. Surfing WebMaster Tool, I found lots of new URL linking at our site: url.org linkarena.com seoprofiler.com folkd.com digitalhome.ca bustingprice.com surepurchase.com lowpricetoday.com oyax.com couponfollow.com aspringcleaning.com pamabuy.com etzone.ca How do I find if those was done intentionelly to hurt SEO? Could it be possible? Thank you, BigBlaze
Intermediate & Advanced SEO | | BigBlaze2050 -
How important is it to clarify URL parameters?
We have a long list of URL parameters in our Google Webmasters account. Currently, the majority are set to 'let googlebot decide.' How important is it to specify exactly what googlebot should do? Would you leave these to 'let googlebot decide' or would you specify how googlebot should treat each parameter?
Intermediate & Advanced SEO | | nicole.healthline0