Crawl questions
-
My first website crawl indicating many issues. I corrected the issues, requested another crawl and received the results. After viewing the excel file I have some questions.
1. There are many pages with missing Titles and Meta Descriptions in the Excel file. An example is http://www.terapvp.com/threads/help-us-decide-on-terapvp-com-logo.25/page-2
That page clearly has a meta description and title. It is a forum thread. My forum software does a solid job of always providing those tags. Why would my crawl report not show this information? This occurs on numerous pages.
2. I believe all my canonical URLs are properly set. My crawl report has 3k+ records, largely due to there being 10 records for many pages. These extra records are various sort orders and style differences for the same page i.e. ?direction=asc.
My need for a crawl report is to provide actionable data so I can easily make SEO improvements to my site where necessary. These extra records don't provide any benefit.
IF the crawl report determined there was not a clear canonical URL, then I could understand. But that is not the case. An example is http://www.terapvp.com/forums/news/ If you look at the source you will clearly see
Where is the benefit to including the 10 other records in the Crawl report which show this same page in various sort orders? Am I missing anything?
3. My robots.txt appropriately blocks many pages that I do not wish to be crawled. What is the benefit to including these many pages in the crawl report?
Perhaps I am over analyzing this report. I have read many articles on SEO, but now that I have found SEOmoz, I can see I will need to "unlearn what I have learned". Many things such as setting meta keyword tags are clearly not helpful. I wish to focus my energy and I was looking to the crawl report as my starting point. Either I am missing something, or the report design needs improvement.
-
Those are commends added to the code. My site has is part of the effort to rid the internet of IE6 browsers. You can read more about it http://www.theie6countdown.com/join-us.html. There is a simple script placed in the site which detects IE 6 & 7 browsers and asks users to update their software.
TYNT is a SEO tool http://www.tynt.com/. It allows webmasters to track any information which is copied and pasted from their site. It creates links back to the site, and tracks all activity on those links.
Both of those scripts are working normally and neither should negatively impact crawls. My Google WMT show my site being crawled normally, no errors. I have multiple pages ranked #1. With that said, there are plenty of opportunities for me to improve, which is why I am here.
The position of the Title and meta tags within the head should not be a factor at all. I did hear Matt Cutts share once that webmasters could move the information to the top to help if there are other issues such as a page where the tag is not properly closed, but that is not really relevant in this case.
-
Your source code looks different to me, if not .... abnormal. What's with doing above the description anyway. I don't know if that code is blocking the crawler from that code to . I'm no code monkey, but all my metas are right next to each other. Yours are completely separated from the title. Take out that code and see if it happens again. OR better yet, go to GWT and FETCH AS GOOGLEBOT. For sure that will tell you what Google bot is seeing.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Medical / Health Content Authority - Content Mix Question
Greetings, I have an interesting challenge for you. Well, I suppose "interesting" is an understatement, but here goes. Our company is a women's health site. However, over the years our content mix has grown to nearly 50/50 between unique health / medical content and general lifestyle/DIY/well being content (non-health). Basically, there is a "great divide" between health and non-health content. As you can imagine, this has put a serious damper on gaining ground with our medical / health organic traffic. It's my understanding that Google does not see us as an authority site with regard to medical / health content since we "have two faces" in the eyes of Google. My recommendation is to create a new domain and separate the content entirely so that one domain is focused exclusively on health / medical while the other focuses on general lifestyle/DIY/well being. Because health / medical pages undergo an additional level of scrutiny per Google - YMYL pages - it seems to me the only way to make serious ground in this hyper-competitive vertical is to be laser targeted with our health/medical content. I see no other way. Am I thinking clearly here, or have I totally gone insane? Thanks in advance for any reply. Kind regards, Eric
Intermediate & Advanced SEO | | Eric_Lifescript0 -
Complicated Duplicate Content Question...but it's fun, so please help.
Quick background: I have a page that is absolutely terrible, but it has links and it's a category page so it ranks. I have a landing page which is significantly - a bizillion times - better, but it is omitted in the search results for the most important query we need. I'm considering switching the content of the two pages, but I have no idea what they will do. I'm not sure if it will cause duplicate content issues or what will happen. Here are the two urls: Terrible page that ranks (not well but it's what comes up eventually) https://kemprugegreen.com/personal-injury/ Far better page that keeps getting omitted: https://kemprugegreen.com/location/tampa/tampa-personal-injury-attorney/ Any suggestions (other than just wait on google to stop omitting the page, because that's just not going to happen) would be greatly appreciated. Thanks, Ruben
Intermediate & Advanced SEO | | KempRugeLawGroup0 -
SEO question
Hi there! I'm the SEO manager for 5 Star Loans. I have 2 city pages running. We are running our business in 2 locations: Berkeley, CA & San Jose, CA. For those offices we've created 2 google listings with separate gmail accounts. Berkeley (http://5starloans.com/berkeley/) ranks well in Berkeley in Gmaps and it shows on first page in organic results. However the second city page San Jose (http://5starloans.com/san-jose/) doesn't show in the Gmaps local pack results and also doesn't rank well in organic results. Both of them have authentic backlinks and reviews. It has been a year already and it's high time we knew the problem 🙂 any comment would be helpful. thanks a lot
Intermediate & Advanced SEO | | moonalev0 -
Best way for Google and Bing not to crawl my /en default english pages
Hi Guys, I just transferred my old site to a new one and now have sub folder TLD's. My default pages from the front end and sitemap don't show /en after www.mysite.com. The only translation i have is in spanish where Google will crawl www.mysite.com/es (spanish). 1. On the SERPS of Google and Bing, every url that is crawled, shows the extra "/en" in my TLD. I find that very weird considering there is no physical /en in my urls. When i select the link it automatically redirects to it's default and natural page (no /en). All canonical tags do not show /en either, ONLY the SERPS. Should robots.txt be updated to "disallow /en"? 2. While i did a site transfer, we have altered some of the category url's in our domain. So we've had a lot of 301 redirects, but while searching specific keywords in the SERPS, the #1 ranked url shows up as our old url that redirects to a 404 page, and our newly created url shows up as #2 that goes to the correct page. Is there anyway to tell Google to stop showing our old url's in the SERP's? And would the "Fetch as Google" option in GWT be a great option to upload all of my url's so Google bots can crawl the right pages only? Direct Message me if you want real examples. THank you so much!
Intermediate & Advanced SEO | | Shawn1240 -
List of Search Engines subscribing to the ajax crawling scheme?
Hi, Does anyone have a list of (major) Search Engines that subscribe to the Ajax Crawling Scheme? (https://developers.google.com/webmasters/ajax-crawling/) Specifically interested in major international Search Engines such as Bing/Yahoo, Baidu & Yandex - if anyone knows, please let me know! Thanks in advance
Intermediate & Advanced SEO | | FashionLux0 -
Seomoz crawl reporting thousands of 302 redirects?
Hi Mozzers I need a little help with Magento redirects. We are using the Ultimo theme [1] and our crawl report has discovered several thousand 302 redirects occurring for links such as: Product comparison links and add to basket links which are formatted as “REPLACE THIS WITH THE INITIAL LINK” are rewritten to links such as “REPLACE THIS WITH THE REWRITTEN LINK ”. Is there any way to stop this from happening? Should I change these rewrites to 301 redirects? Or is this perfectly normal? [1] - http://themeforest.net/item/ultimo-fluid-responsive-magento-theme/3231798 Thanks in advance @Anthony_Mac85
Intermediate & Advanced SEO | | Tone_Agency0 -
Looking to merge 2 domains to 1 - Got some questions - do you have the insight?
Hey Everyone, So the company I work for owns 2 domains. We have our main site which offers our portfolio of products and then we have a second domain, which we acquired, which focuses on one of our products (we also have this product available on our main site). So here is where things get tricky for me. This second site (the one that focuses on one of our products) has a HUGE following and a higher domain authority of 80 when our main site has an authority of 70. The higher up of the company want to merge the second popular site with our main site. There are many problems with this in my opinion since the following on this second site is very hardcore in the security space (I do not think that they will like to be sent over to a more corporate site) BUT I want to figure out the SEO value that can be gained or lost from this merge. Some questions... By 301 redirecting the pages over to our main page - I am assuming that the SEO power carries along with it so that these pages should still perform well? Will the domain authority of our main site go up with the merge since we are bringing over pages with a lot of equity? In your opinion, does it make more sense to keep the site with the higher authority since it is easier to host content that performs better? Anyone have any experience with this? SEO-wise do you think that this is a good idea or a bad idea? Thanks a lot! Pat
Intermediate & Advanced SEO | | PatBausemer0 -
Sitemap.xml Question
I am pretty new to SEO and I have been creating new pages for our website for niche terms. Should I include ALL pages on our website in the sitemap.xml or should I only have our "main" pages listed on the sitemap.xml file? Thanks
Intermediate & Advanced SEO | | threebiz0