Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
What is the best tool to view your page as Googlebot?
-
Our site was done with asp.net and a lot of scripting. I want to see what Google can see and what it can't. What is the best tool that duplicates Googlebot? I have found several but they seem old or inaccurate.
-
I just used it today. The simple crawl is free. Advanced is paid.
-
I think it's free again? I use it today...
-
Used to be free, now it's 9,99$ per year
certainly not unreasonable, but some sort of demo would be nice so one knows what to expect. Do you have another recommendation? -
Hi Niners52,
Personally, I like to use webconfs search engine spider tools (http://www.webconfs.com/search-engine-spider-simulator.php) but there are loads out there that you can use and it is your own preference really.
Also, I use ScreamingFrog to do similar tasks.
Matt.
-
I'm a fan of http://www.seo-browser.com/ - use their simple search and it's totally free.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Reason for robots.txt file blocking products on category pages?
Hi I have a website with thosands of products. On the category pages, all the products are linked to with the code “?cgid” in the URL. But “?cgid” is also blocked in the robots.txt file for some reason. So I'm thinking it's stopping all my products getting crawled by Google. Am I right here? Is there any reason why a website would want to limit so many URL's? I'm only here a week and the sites getting great traffic, so don't want to go breaking it!!! Thanks
Web Design | | Frankie-BTDublin0 -
Problems preventing Wordpress attachment pages from being indexed and from being seen as duplicate content.
Hi According to a Moz Crawl, it looks like the Wordpress attachment pages from all image uploads are being indexed and seen as duplicate content..or..is it the Yoast sitemap causing it? I see 2 options in SEO Yoast: Redirect attachment URLs to parent post URL. Media...Meta Robots: noindex, follow I set it to (1) initially which didn't resolve the problem. Then I set it to option (2) so that all images won't be indexed but search engines would still associate those images with their relevant posts and pages. However, I understand what both of these options (1) and (2) mean, but because I chose option 2, will that mean all of the images on the website won't stand a chance of being indexed in search engines and Google Images etc? As far as duplicate content goes, search engines can get confused and there are 2 ways for search engines
Web Design | | SEOguy1
to reach the correct page content destination. But when eg Google makes the wrong choice a portion of traffic drops off (is lost hence errors) which then leaves the searcher frustrated, and this affects the seo and ranking of the site which worsens with time. My goal here is - I would like all of the web images to be indexed by Google, and for all of the image attachment pages to not be indexed at all (Moz shows the image attachment pages as duplicates and the referring site causing this is the sitemap url which Yoast creates) ; that sitemap url has been submitted to the search engines already and I will resubmit once I can resolve the attachment pages issues.. Please can you advise. Thanks.0 -
Internal Linking: What is the best practice for pages not included in Nav bar?
I never quite understood why internal linking was such a big deal for SEO, but now I'm having second thoughts and perhaps understanding it more. I always thought since most websites have a navigation feature--usually the menu bar located at the top and often another one in the footer--that internal navigation was usually already built in to most websites and therefore, a silly topic to make a fuss over; however, I may be the silly one after all. I am now creating pages that are not included in the navigation so.... What is the best practice for this? If I am creating say, pages for certain locations and those location pages begin to number in the hundreds, it makes my navigation bar a little too cumbersome to have all those pages in a drop down menu. So I made a Locations page and just link to all those pages from that page (and from nowhere else). But now I'm wondering if this could be a bad internal linking practice and perhaps hurt my online visibility as an SEO ranking factor. Is this a crawl problem? And if so, is there a better option that provides a good visitor experience while appeasing the search engines.
Web Design | | Dino640 -
Duplicate Content Issue: Mobile vs. Desktop View
Setting aside my personal issue with Google's favoritism for Responsive websites, which I believe doesn't always provide the best user experience, I have a question regarding duplicate content... I created a section of a Wordpress web page (using Visual Composer) that shows differently on mobile than it does on desktop view. This section has the same content for both views, but is formatted differently to give a better user experience on mobile devices. I did this by creating two different text elements, formatted differently, but containing the same content. The problem is that both sections appear in the source code of the page. According to Google, does that mean I have duplicate content on this page?
Web Design | | Dino640 -
Too Many Outbound Links on the Home Page - Bad for SEO?
Hello Again Moz community, This is my last Q of the day: I have a LOT of outbound links on the home page of www.web3.ca Some are to clients projects, most are to other pages on the website. Can reducing this to the core pages have a positive impact on SEO? Thanks, Anton
Web Design | | Web3Marketing870 -
Does it do harm if you add a rel="canonical" tag on a page that doesn't need it?
If a page is clearly unique and there is obviously no canonical tag needed, does it hurt anything if one has been added?
Web Design | | jaychow0 -
Html 5 main and secondary navigation for SEO best performances
I am building a website which will have a main navigation related to the site and each link of the main navigation will have a secondary navigation. We do not want to use a megamenu style navigation. I will try to explain it with a example: Let's start with an example for a computer store "My PC Store", the Main Navigation would be: Desktop PC's Notebook & Tablets
Web Design | | netbuilder
Multimedia When clicking on the "Notebook & Tablets" the user is directed to the page domain.com/notebook-tablet.html and on this page the secondary navigation appears: Laptop Netbook Tablets / iPad I am confused on how I should organize the semantic navigation for best SEO performances and I need advice / suggestions. I thought about 2 different ways to do it but which one is more appropriate in terms of SEO? PROPOSITION A Home Page: <header> My PC Store <nav> Desktop PC's Notebook & Tablets Multimedia </nav> </header> Sub-Page (Notebook & Tablets): <nav>(or <aside>?) Desktop PC's Notebook & Tablets Multimedia </aside> </nav> <header> Notebook & Tablets <nav> Laptop Netbook Tablets / iPad </nav> </header> As you notice on the home page the Main Site Navigation is included in the <header>while it is not in the sub-pages. PROPOSITION B Home Page: <header> My PC Store <nav> Desktop PC's Notebook & Tablets Multimedia </nav> </header> Sub-Page (Notebook & Tablets): <header> Notebook & Tablets <nav> Desktop PC's Notebook & Tablets Multimedia </nav> # Notebook & Tablets * Laptop Netbook Tablets / iPad </header> The main navgation remains always in the <header>(home page / sub-pages) of all page. I need suggestions... How would you guys organize the nav ? </header> </header>0 -
Custom 404 Page Indexing
Hi - We created a custom 404 page based on SEOMoz recommendations. But.... the page seems to be receiving traffic via organic search. Does it make more sense to set this page as "noindex" by its metatag?
Web Design | | sftravel0