Welcome to the Q&A Forum

Hi Dwight,

The crawl did change a bit. Check out this thread http://www.seomoz.org/q/domain-authority-decrease-after-open-site-explorer-update-reasons for some more information from the OSE people.

Hi Diane,

I checked with the help desk, and they looked at the questions associated with your profile email. There was a reply from the help team to your most recent question, but maybe you didn't receive that? If you can give us the ticket number you got with your initial request, we can look that up and get back to you. The email from the help desk would have zendesk in the domain, which might help with a search to see if that email got filtered somehow.

For questions, we did have a problem for about two days where people could submit both private and public questions and they wouldn't be displayed, even to staff. This happened at the same time as the new OpenSiteExplorer was released and MozCon started, so it took a little longer than usual to reply. I do see a question from two days ago that had a response written earlier this morning. If this is not the question you're referring to, please send me a private message and I'll investigate further. Private questions can take a few days for a response, and combined with the Q&A being down stuff did get a little slower. We're so sorry about that, and Associates are going to be working this weekend to help clear up the backlog.

Keri Morgret

SEOmoz Associate

I wouldn't quite say that Roger was slacking, but more like he was running in a bunch of different directions at once. It's unfortunate that the Q&A system had problems right when we started MozCon and the new OSE was rolled out. All of the questions did get posted, and the Moz team is looking through the questions that got posted. So sorry for the down time!

Hi! Yes, there was a problem, but it's all fixed now. There's some backlog, but you should start seeing new questions make it to the web. So sorry for the downtime!

Hi! It is a result of the OSE update. Rand talks about it a little in the blog post at http://www.seomoz.org/blog/brand-new-open-site-explorer-is-here when he says:

In addition to all the amazing new features in Open Site Explorer, Linkscape's index just updated using a new infrastructure that's allowed us to crawl much deeper on large, important sites. For many pages/domains, this will mean an increase in the total number of links we report, but likely a lower count of linking domains (unless you've gained a lot of links in late June/July) since we're excluding many domains that are low-quality/not-well-linked-to. We'd love your feedback on this index, as it's the first one of its kind, and will continue to see tweaks/improvements over the next few updates.

Hi! Don't panic! This is part of the OSE algorithm update. The OSE engineers will be over here in the next few days to give some more explanation here, but wanted to let you know now that it isn't your website that had a big change.

Quick test, please ignore.

Hi Cees,

I would actually wait until Tuesday (US time) and re-run your metrics. Open Site Explorer is updating as I type this, so in another few hours there should be fresh information. Run your reports and see if there's still the same problem. If there is still the problem, open a ticket with the help desk either through the web or by emailing help@seomoz.org.

Thanks!

This sounds like a case for the help desk. Email them at help@seomoz.org with details of your problem. Also include a link to this Q&A thread and mention that one other person has gotten the error occasionally, so the help desk can follow up with Per Allerup if they need any additional information.

Thanks!

Good news! Wordpress.com now allows you to purchase a 301 redirect to an external site. Information is available at http://en.blog.wordpress.com/2010/10/04/offsite-redirect/ and http://en.support.wordpress.com/site-redirect/.

Does the server return a 503 not available when it goes down at least? Or is it somehow set to show a 404? You want to make sure it's showing a 503 to the bots so they know the server is down and not that the pages have been removed.

I couldn't find it in the first 15 pages of Google either. It could be that it's someone in your social circle that has shared it and that's what is pushing the results up higher.

One thing I like to do before I panic too much is fire up Safari, go to Edit -> Reset Safari to set everything to like it's a fresh install (clear cookies, history, etc. in one easy step), then search when I know that I have a minimum amount of personalization.

I agree with the thin content assessment. Below is what I wrote before I read your comment a second time.

I'm looking at interior pages on the site, and my question is "if I were Google, what benefit do I get from including this site in the index?". What does this site offer that's not already on the web? I visited a few random pages. Several of the offers were expired, some as much as a couple of years ago. The product information is taken from another site, and is duplicated all across the web. The blog hasn't had a new post since January. There is very little to this site that is not from a data feed, from what I can tell. Why should Google include this site in their results, when other sites can offer more information?

On the technical side, don't include "all" in the robots meta tag, it's not an option (see http://www.robotstxt.org/meta.html). I don't know if it's causing any problems, but it doesn't need to be there.

Google made a post in their Webmaster Central blog about what makes a quality site, and offered a number of questions for people to think about in relation to a site or a page on a site. http://googlewebmastercentral.blogspot.com/2011/05/more-guidance-on-building-high-quality.html

One of the questions was "Does this article have an excessive amount of ads that distract from or interfere with the main content?" which would make me think they look at ads to some degree.

Hi Dunamis,

Check out the post Justin Briggs made about How to Build Links with infographics. He does talk about the landing page for an infographic, and mentions this in relation to one specific example.

This landing page removes almost all distractions, except for engaging with the content via social or embedding. It links back to the post and the site’s homepage. There is this wonderful share bar at the top, which allows visitors to easily share the graphic. It even sticks at the top as the user scrolls down the page.

If you look at the link for this example, the link goes directly to the infographic with almost nothing else on the page except for social sharing. It does link back to the infographic in the context of a post, where the site looks more like a regular page on the site.

Hope this gives you an idea of how other people balance the layout options you have.

Hi Jay,

Here's a page the help desk wrote about why your site may only have one page crawled. Your best bet is to read that, then if you're still having problems, open up a ticket with the help desk either via the web or by sending an email to help@seomoz.org.

http://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-only-crawled-one-page

Thanks!

It's not new. You can read some more information about when it first was reported in late 2008 over at Search Engine Roundtable. http://www.seroundtable.com/archives/018996.html.

YOUmoz might be a good place to write up those results, too!

You build the links with human contact. Once you've identified a site with a tool, look for yourself to make sure it's relevant, and then craft an email that is designed for that site. If you send an automated link request to my site about rc combat ships and you want me to link to window blinds, you're not going to get a response.

It could be something from Joomla, like it's creating printable versions of the pages. Are you still having this issue? Do the reports of duplicate pages give you any clues form the URL?

SEOmoz and Google Webmaster Tools use different sources for their data. I believe EGOL meant to look at the trends over time for each metric compared to itself. Don't compare GWT to SEOmoz, compare GWT in one month to GWT in the next month, and SEOmoz in one month to SEOmoz in the next month.

Open Site Explorer updates about once a month, so it does have a lag time in showing your updates, and it only crawls about the top 1/4 of the web, so its numbers will differ from GWT.

Theo, did you ever find any more information on this? Would love to see anything you did find, as it is a topic others have asked about. Thanks!

Hi Dustin,

Do you know if any code changes have been made? Maybe accidentally something modified the robots.txt and excluded most of your site? Has your organic traffic from Bing and Yahoo had a similar drop? Before getting the tin foil hat on too tight, check the robots.txt file and the code on your page and make sure something didn't go astray there.

Can you share your URL with us?

Agreed that  I would not dare use it either. You couldn't pay me enough to be part of that type of link exchange, even if all 5400 backlinks were dedicated to my exact niche (not that that many sites in my niche even exist!). If you look at the telalinks site more, you can see they do a poor job of maintaining their own site. There are multiple 404 pages when you click for an example of what a links page would look like, and in a random clicking of some of their sites in the advertising section I found a male porn site, a Chinese medicine site, and designer handbags. If they pay this little attention to their own site, I am really not going to trust what they put into a 1k text file for me to upload.

I think because that section is labeled "crawl errors", an area blocked from crawling would be considered an error. I can see where you're coming from, but think of it as an error found with an attempt to crawl, not necessarily an error found in the site itself.

One advantage of using the first type of URL (the absolute URL) is that if people scrape (copy) your content, the links within the content will go back to your site.

Hi Morch, I'm wondering what you decided to do, and if you've seen any more changes in your rankings. Do you have any advice to pass along to someone else who may be in the same situation, or any other questions to ask?

Check out this thread from earlier this month, where someone was evaluating SENuke and decided against it. You can read his experience and the opinion of other people as well. Generally, it was not a positive opinion.

http://www.seomoz.org/q/se-nuke-is-it-any-good

Use relative paths, and you lose all link juice when someone scrapes your content and reposts it.

And Google has just discontinued their directory, so the world will never know. Do let us know when you see it pop up in search.com, it'll be interesting to know how long that takes. Here's the article about Google Directory going down. http://searchengineland.com/final-nail-in-the-google-directory-coffin-86505

Google just discontinued the Google Directory, so it's now a bit of a moot point. There's a post at http://searchengineland.com/final-nail-in-the-google-directory-coffin-86505 with more information.

This may be more of a moot point now, as Google Directory has been discontinued. There's a post at http://searchengineland.com/final-nail-in-the-google-directory-coffin-86505 about this.

And now Google has shut down the Google Directory, so it's just DMOZ. http://searchengineland.com/final-nail-in-the-google-directory-coffin-86505

It's been four months since the original question. I'm now seeing your site rank on the second page (#14 overall) for beauty tips on Bing. Have you made changes, or do you have any tips you can pass on for how you got ranked?

Hi Kathryn,

I see there are now some more recent blog posts in the index, including from this month. Were you able to figure out why there was this reduction? Did you make changes, or did things just start reappearing on their own? The dates on most of the items in the index look older, but there are at least a few newer items in the index now. Is there anything you'd like us to take a fresh look at?

Blueglass has had a lot of posts about building infographics. You might start with http://www.blueglass.com/blog/what-makes-a-good-infographic/ then do a search on their site for infographics to find their other posts.

Hi Corey,

This should answer your question. It's written for Wordpress, but the technique is the same for most any site with an editing software. http://en.support.wordpress.com/links/

Run an SEOmoz campaign and look there for the referring URLs?

You might also consider writing the redirect so that you don't have any extensions (see the SEOmoz URL for an example). That way, whenever you next change technologies, your URL will still stay the same.

If you're going to redo your URLs, you might look at this post by Lunametrics about Google Analytics friendly sites. It offers a few things to think about so your URLs can also provide you value in your analytics.

http://www.lunametrics.com/blog/2010/09/22/designing-google-analytics-friendly-site/

You should be able to see the referring URL for the 404 pages in your GWT account. If someone links to you and does a typo on the end of the URL, or puts your URL in parenthesis and the final ) gets hyperlinked, that can cause a 404.

The short answer is that yeah, it's a Google thing. Google has been changing titles a lot more lately, and has often tweaked the meta description. Barry has a post on it over at Search Engine Roundtable. http://www.seroundtable.com/google-title-tag-13704.html.

You don't have anything set up wrong in your CMS, it's Google. It's late, so I don't have a longer answer for you, but hopefully this will help keep you from pulling out any more hair while we wait for some more people to chime in.

Do you know how many pages the client is supposed to have? If they have 100 or so pages, one guess would be that there's an issue with URL parameters creating a ton of duplicate content, or there's a calendar application that's causing endless pages.

As it's a client, I imagine you can't share the URL with us. If you're not seeing any messages in your campaign about duplicate content or other hints, I'd email help@seomoz.org and ask them to take a look at it.

Thanks for the update! It's always interesting to hear what people ended up doing. If you're interested, consider writing up a post for YOUmoz when all of this is done. Martin is right, it's a post waiting to happen!

I haven't found any list that's newer than a year old in the brief search I did. I did find these comprehensive user agent lists at http://www.botsvsbrowsers.com/ and http://www.user-agents.org/.

Are you more interested in every variation of bot there is, or more after the top 20 bots?