Does Rogerbot recognize rel="alternate" hreflang="x"?
-
Rogerbot just completed its first crawl and is reporting all kinds of duplicate content - both page content and meta title/description.
The pages it is calling duplicate are used with rel="alternate" hreflang="x", but are still being labeled as dupes.
The title and descriptions are usually exactly the same, so I am working on getting at least those translated into different languages.
I think its getting tripped up because the product page its crawling are only in English, but the chrome of the site is in the translated languages. The URLs look like so:
Original: site.com/product
Detected duplicates: site.com/fr/product, site.com/de/product, site.com/zh-hans/product
-
Hey there,
Rogerbot doesn't look for rel alts. The bot will follow meta robots, rel canonical (more used way to controlling duplicate content) and 301 redirects. Sorry about the confusion.
Best,
Nick
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Title "Untitled Document" with mozbar
Hi, I'm working on a big news website (in tourism) and, when checking my homepage with mozbar, I often get: "My Title • Untitled Document" - 69 characters With no apparent reason, If I refresh the page or simply visit it later, it could randomly switch to the "normal" title: "My Title" - 44 characters It looks like completely random to me, like something more is loaded in the first case. I've checked my source code of course, and <title>tag seems to be ok with no other <title> even when I get the "untitled document".</p> <p>I have no idea about that. I don't know if that can be a useful info, but I was trying to understand why the majority of my main keywords searches are only showing other pages (I can rarely find my homepage) even though my home looks correctly indexes and shows as first result for every site: research.<br /><br />Hope everything's is clear....any idea or suggestions?<br /><br />Thank you in advance.</p> <p> </p></title>
Moz Pro | | Daniele_Carollo0 -
Rogerbot crawls my site and causes error as it uses urls that don't exist
Whenever the rogerbot comes back to my site for a crawl it seems to want to crawl urls that dont exist and thus causes errors to be reported... Example:- The correct url is as follows: /vw-baywindow/cab_door_slide_door_tailgate_engine_lid_parts/cab_door_seals/genuine_vw_brazil_cab_door_rubber_68-79_10330/ But it seems to want to crawl the following: /vw-baywindow/cab_door_slide_door_tailgate_engine_lid_parts/cab_door_seals/genuine_vw_brazil_cab_door_rubber_68-79_10330/?id=10330 This format doesn't exist anywhere and never has so I have no idea where its getting this url format from The user agent details I get are as follows: IP ADDRESS: 107.22.107.114
Moz Pro | | spiralsites
USER AGENT: rogerbot/1.0 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+pr1-crawler-17@moz.com)0 -
Data Update for RogerBot
Hi, I noticed that rogerbot still give me 404 for http://www.salustore.com/capelli/nanogen-acquamatch.html refferal form http://www.salustore.com/protocollo-nanogen even I made changes since a couple of week. Same error with one "Title Element Too Short" on our site. Any suggestion on how to refresh it? Best Regards n.
Moz Pro | | nicolobottazzi0 -
How can I recognize spammy links that harm my website
Hello, Can someone tell me how I can recognize links that can harm my website? I have OpenSiteExplorer and CognitiveSEO as tools to create nice backlinking reports for me. But my problem is, how do I know if a link shown on these reports can harm my website so I can remove them? Is it linktypes, low auth. links etc? Thank you for your answers in advance! Regards, Thomas
Moz Pro | | ThomasH0 -
Campaign status stock in status "Next Crawl in Progress!
Has anyone else had an issue laetly where the campaign status was stock in _ **"Next Crawl in Progress!" **_? One of our campaigns has been in this status for the page 2 1/2 days and this has not happened in the past as there are only 597 pages for this campaign to crawl. I send a help ticket request to the SEOMOZ team but was wondering if this is an isolated issue or if other community members have also experienced it? Thanks.
Moz Pro | | DRTBA0 -
Whats rel canonical
I have a warning in SEOmoz saying that I have 150 rel canonical - What the hell that means? 🙂 Tks in advance 🙂 Pedro Pereira
Moz Pro | | PedroM0 -
Anyway to Verify MozRank & MozTrust Scores w/alternate method?
Hey everyone, I am wondering if there is a way to confirm or reinforce what am i seeing on a webpage using the Moz Toolbar? When I go to this website here: abilogic.com they have a high page/domain trust rank. However, when I filter down to the niche category I would be interested in submitting too abilogic.com/dir/Home/Gardens/ - Then everything, including PageRank, drops to zero. The question I have, is I am not sure if this is really just a poorly constructed and untrustworthy page as identified in the Moz Toolbar. Or is it possible that these page categories have recently been redeveloped or something like that, and that's why the deeper pages are not showing any benefits? Thanks for any help or tips!
Moz Pro | | JerDoggMckoy0 -
SEOmoz bot and "noindex"
As a recent newbie to SEOmoz, I've been implementing some suggestions and doing a general tidy up. I removed URL's from our robots txt, and rolled out instead the noindex meta tag to pages we don't want indexed. But surprised to see issues that are now flagged from the last crawl by the moz bot from pages that have this meta tag? Does the SEOmoz bot not ignore this tag? Just want to make sure I've implemented it correctly, so the google bot does ignore it. Meta tag syntax is and is placed below the title tag. cheers Steve
Moz Pro | | sjr4x40