Roger returned single page
-
A search query in Google shows 156 indexed results. Yet Roger has completed a crawl and returned a single page. The website is table based, but there appear to be no redirects or Javascript blocking bots so I'm unsure why Roger has under delivered. The problem is not unique to Roger. I ran the site through Screaming Frog's desktop crawler and it also returned a single page.
I'm wondering if there is something in the site code I don't know to look for that is stopping Roger & Screaming Frog from crawling the site.
Appreciate any insights you can offer.
PS. I've read https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages- and don't think these suggested causes apply.
-
Hi Sam, have PM'd you.
-
Hi Nic,
I haven't heard back from you so I wanted to pop in really quickly. I went to the site associated with your email and if that's the one you're talking about, the reason why no one is able to crawl it is because there is no link. It looks like you may want to keep this private though so if that's the case, feel cree to reach out to us directly via help[at]moz.com. If you'd like to continue this conversation in the community, feel free to update this thread with the URL in question. Thanks!
Best,
Sam
Moz Helpster -
Sounds like a redirect to me Moz & Screaming Frog's desktop crawler and it also returned a single page.
Hope that helps,
Thomas
-
Hi Nic!
I'd be happy to take a look, but I'm not sure what URL you're referring to. I reviewed your campaigns and crawl tests and didn't see anything that was stuck on a single page crawl. Can you tell me either which campaign this is for or the URL in question? Looking forward to hearing back from you!
Best,
Sam
Moz Helpster
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Dropping pages from SERPS?
Our website for my olansi company in London, China has hundreds of pages dedicated to every service we provide to China local areas. The total number of pages is approximately 100. Google caters pretty well for long-tail searches when it indexes all these pages, so we usually get a fair amount of traffic when this happens. However, Google occasionally drops most of our indexed pages from search engine results for a few days or weeks at a time - for example, Google is currently indexing 60 pages while last week it was back at 100. Can you tell me why this happens? When these pages don't display, we lose a lot of organic traffic. What are we doing wrong? Site url:https://www.olanside.com
Technical SEO | | sesahoda0 -
Very wierd pages. 2900 403 errors in page crawl for a site that only has 140 pages.
Hi there, I just made a crawl of the website of one of my clients with the crawl tool from moz. I have 2900 403 errors and there is only 140 pages on the website. I will give an exemple of what the crawl error gives me. | http://www.mysite.com/en/www.mysite.com/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | | | | | | | | | | There are 2900 pages like this. I have tried visiting the pages and they work, but they are only html pages without CSS. Can you guys help me to see what the problems is. We have experienced huge drops in traffic since Septembre.
Technical SEO | | H.M.N.0 -
Home page URL
Hi, I work on this site: http://www.towerhousetraining.co.uk/about-us. This is the home page URL. Should this be 301'd to: http://www.towerhousetraining.co.uk? I have created a site map, which I submitted to Google Webmaster Tools, which includes these URL's: /about-us, /training-we-offer & /contact-us. There are a total of 3 pages on the website. Webmaster tools has only indexed 2 out of 3 pages. I think this is something to do with the /about-us URL, as when I do a site: search, these pages appear: www.towerhousetraining.co.uk/, /training-we-offer & /contact-us. I am not sure why Google has indexed the home page as www.towerhousetraining.co.uk/ and not /about-us? Is it a bad idea in general not to have your homepage as your root domain? I added a to the homepage, but am wondering if this was the right thing to do? Any help would be appreciated.
Technical SEO | | CWseo0 -
My website pages are not crawled, what to do?
Hi all. I have made some changes on the website so i like to crawled them by the search engines Google especially. I have made these changes around 2 weeks ago. I have submitted my website on good bookmarking websites. Also i used a tool available in Google webmasters "Fetch as Google", Resubmitted a sitemap.xml. Still my pages are not crawled your opinion please. Thanks
Technical SEO | | lucidsoftech0 -
Drupal duplicate pages
Anyone else encountered massive numbers of duplicate pages being reported on SEO Moz crawls for Drupal based sites? I assumed it was b/c there was no redirect on the print format pages, so I fixed that with a cannonical tag. But still seeing 2 or 3 duplicate pages reported for many pages. Any experience fixing this would be awesome to hear about. Thanks, Kevin
Technical SEO | | kevgrand0 -
Wordpress Archive pages
In the SEOMOZ site report a number of errors were found. One of which was no or duplicate meta desctions on certain blog pages. When I drilled down to find these i noticed thosepages are the wordpress autocreated archive pages. When I searched for these through the wordpress control panel through both pages and blogs they were nowhere to be found. Does anyone know how to find these pages or are they not something I need to worry about?
Technical SEO | | laserclinics0 -
SEOMoz Crawl Diagnostic indicates duplicate page content for home page?
My first SEOMoz Crawl Diagnostic report for my website indicates duplicate page content for my home page. It lists the home page URL Page Title and URL twice. How do I go about diagnosing this? Is the problem related to the following code that is in my .htaccess file? (The purpose of the code was to redirect any non "www" backlink referrals to the "www" version of the domain.) RewriteCond %{HTTP_HOST} ^whatever.com [NC]
Technical SEO | | Linesides
RewriteRule ^(.*)$ http://www.whatever.com/$1 [L,R=301] Should I get rid of the "http" reference in the second line? Related to this is a notice in the "Crawl Notices Found" -- "301 Permanent redirect" which shows my home page title as "http://whatever.com" and shows the redirect address as http://http://www.whatever.com/ I'm guessing this problem is again related to the redirect code I'm using. Also... The report indicates duplicate content for those links that have different parameters added to the URL i.e. http://www.whatever.com?marker=Blah Blah&markerzoom=13 If I set up a canonical reference for the page, will this fix this? Thank you.0 -
Page not Accesible for crawler in on-page report
Hi All, We started using SEOMoz this week and ran into an issue regarding the crawler access in the on-page report module. The attached screen shot shows that the HTTP status is 200 but SEOMoz still says that the page is not accessible for crawlers. What could this be? Page in question
Technical SEO | | TiasNimbas
http://www.tiasnimbas.edu/Executive_MBA/pgeId=307 Regards, Coen SEOMoz.png0
