Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
the search bar results are good for searchers but not for search engines. You can stop all search engines and Roger (the seomoz crawler) from going into those pages by adding an entry to your robots.txt file. Roger only responds to his own section of the robots file, so anything you make global will not work for him.
User-agent: rogerbot Disallow: /search/*
I think this is venturing into dangerous ground. Imagine if thousands of anti-Rush Limbaugh fans put his site into their block list and google stopped displaying his site, then all his fans did the same to Obama We would end up with people blocking their competition
Site looks nice, Paul. If you haven't done this already, use the SeoMoz tool to check your onpage setup: http://pro.seomoz.org/tools/on-page-keyword-optimization/new Fix anything that needs work and then start promoting your site. You can set up tracking here: http://www.seomoz.org/rank-tracker You will be emailed regular updates, that show if you are making progress. As it is a real site, issue a press release every week, about something interesting you are doing or that is of interest to people buying travel insurance in Ireland. Change which pr site you use, so they aren't all from the same one. Write other articles and get them published in other places - offer them to your local news sites.
Cameron, I always recommend businesses link to their press releases. Doing so provides value in different ways. First, if there are no links and no references to it, it basically has no value, Second, you get a little bit of third party validation from it. Third, you can get visitors from it and brand recognition. I know of releases that we published in 2005 that still get hundreds or thousands of visitors each month, because they have incoming links. Don't base all your decisions on what immediate link juice you might get from an action you take, because it may hurt you some time in the future. And yes, prlog is good. I have some I will send there this week.
I believe it is much better to allow the search engines to handle it themselves. Check your logs to see what they are doing, so you know if they are hammering you server. If they are, you can slow them down using the robots.txt file On a large site, google will often such out a few hundred thousand pages per day. Yahoo will do more, but their crawler isn't always friendly or smart. Sometimes different Slurp crawlers will request the same pages. Bing emailed me six months ago and asked that we remove the 2second delay I added. I did that because they were hammering the server, but only sent a few thousand readers per day. Google doesn't usually hammer the server, but when 5 or 6 are hitting you at the same time, it can affect you site response time. - and then some of them want to penalize you for having a poor experience for visitors.
Cameron, You have services to offer, so you should also consider writing press releases about your company and your services. There are many free press release sites as well as paid ones. Most allow a link back to your site. Most allow you to use link text. Localize your releases and you should get visitors from them. make your releases interesting. Structure them well and they will help you These are one-way links.
Hello Alan.
No, I don't think so.
It is the wrong pattern for panda, from what I have seen.
At least you are showing up in the results.
I believe if it was panda, that you would have to go to the end of the displayed results and click on the link that displays all of the results, including the results that were previously hidden.
Then you would show up, in a relatively high position.
The change you describe doesn't fit that pattern.
What I described is the only pattern I think is panda.
OK I see what you mean, Robert - that is a great idea.
I'm a programmer too, so a call to me isn't a phone call, it is a call of a program, in this case a direct access - from someone who typed in the url or has the site bookmarked. I'm mixing up my nomenclature, sorry for the confusion.
yes, that was horrible.
If they tested it first, it must have been with a bunch of goth geeks
Thank you Keri.
Yes, good idea, but whatever you request, that page or directory must respond with a 404, otherwise, it will be ignored.
(would have been a nice thing to do)
I guess I could have cheated, and made them return a 404 if it was google, just to dump them all out of the index.
The 15,000 I did request to be removed were individual pages, that returned 404 response code, so thats why I did them one at a time. I could have waited, but if you wait, then google keeps trying to fetch those missing pages and they keep reporting them in your GWT.
That is a good reason to request the removals.
I actually gave up when the number of deletions got to 1.5 million. I figured it was just too hard to do.
I would have said the same thing, except that a few weeks ago, I removed a rule from the robots file and I changed the affected pages to have a noindex.nofollow and the next day, tens of thousands of those pages appeared in the index and overpowered the content pages.
So my advice, is don't trust noindex,nofollow and just stop the robot going down that tree (as you are doing) and find another way to get those pages out of the index.
You can use the URL removal request tool.
It only seems to allow you to remove 1000 per day.
I have done this before by automating the removal using a macro program.
I think I removed about 15,000 over the space of a month, doing that.
They are fairly fast at removing URLs these days, 24 hours or less.
Hello Colt.
You have the joys of a Microsoft webserver.
That means all of these URLs work - and many more:
http://www.federalmortgageservices.com/A-Home-Loan-Is-Possible-with-Bad-Credit.aspx
http://www.federalmortgageservices.com/A-Home-Loan-Is-Possible-with-Bad-CRedit.aspx
http://www.federalmortgageservices.com/A-Home-Loan-Is-Possible-with-Bad-CREdit.aspx
http://www.federalmortgageservices.com/A-Home-Loan-Is-Possible-with-Bad-CREDit.aspx
It does exactly the same thing if you remove the www.
Also, if you do this:
http://www.federalmortgageservices.com/A-Home-Loan-Is-Possible-with-Bad-Credit.aspxZ
I didn't find a way to make your server give me a 404
The bad things about programming extensions, php, pl, asp and others are:
- that you are telling people the code on your site
- that you are adding one more thing that people need to remember
- you are obfuscating your link name
As josh says, it doesn't make any difference to the search engines, but the lack of the extension is cleaner.
Hello Sudesh
http://abhishekyadav.com/ responds with a 200 response code
but then the page it serves up is an immediate META refresh to
http://abhishekyadav.com/cgi-sys/defaultwebpage.cgi
and that fails
Your main URL without www sends a 301 redirect to the domain with the www
At the time google crawled your site, maybe your redirects were messed up and they think you moved to abhishekyadav
I think you should fix that meta refresh and probably 301 redirect instead to www.replicahause.net
You should be able to go into your webmaster tools and tell the google crawler your site has moved from
abhishekyadav to replicahause
Here is another way you could handle it, in a separate program that detects a smartphone.
For a phone that can handle it, you do a redirect and for one that doesn't, you do something else
(obviously, I haven't tested this, but this way also has the upside that it tracks the clicks.
If your site uses this style of URL
Call 555 555 5555
Or this if you handle URLs this way
Call 555 555 5555
Sorry I don't know what you mean about adding a number?
We haven't changed anything other than continuing to add 30 to 50 stories per day.
No internal structure has changed. We don't do any "link building" - all our incoming links come from people linking to content on their own.
Hi EugeneF
The problem with tel: is that most browsers don't know what to do with that, so they see it as a URL relative to your site, and when you try http://www.google.com/tel:1231231234 there is no such thing and of course, you get a 404 error.
So here is another gotcha that we will all have to cater for in our robots files and .htaccess files.
What you need to do, to handle that, is to detect that you are dealing with a smart browser that understands what to do with it, and only display those anchor links to those browsers.
The upside, of course, is that either robots or hopefully, real people are clicking on your phone links.
The downside is that if they are real people and they get a 404 error, you are giving them a bad surfing experience.
Thank you István
I added this:
User-agent: rogerbot
Disallow: /sendtoafriend/
Disallow: /photo/
Disallow: /pix/
Is what I added OK?
Thank you Robert.
Because it is a big news site, things are always changing, but we haven't made any big changes.
no new major sources of referrals.
Most other traffic is relatively steady.
Direct calls slightly increasing.
We don't do any PPC
It isn't anything to do with terms, we don't really do that.
If it shows 0 root domains, it could mean many things.
Those links were not in place, the last time the link check was done
If they were in place, it doesn't know about them
There is always a lag, no matter how good the links are.
PR has nothing to do with relevance for keywords.
PR just gives you a warm fuzzy feeling (or not)
I was so happy when my Moz Points reached 100 today, but I've done a whole lot more today and it still says 100.
Have I broken the system, or is it resting today (it is Sunday where I am)
Google tells us to make our sites better for our readers, which we are doing, but do you think google has horrible site usabilty?
For example, in webmaster tools, I'm always being confused by their changes and the way they just drop things.
In the HTML suggestions area, they don't tell you when the data was last updated, so the only way to tell is to download the files and check.
In the URL removals, they used to show you the URLs they had removed. Now that is gone and the only way you can check is to try adding one.
We don't have any URL parameters, so any parameters are as a result of some other site tacking on stuff at the end of our URL and there is no way to tell them that we don't have any parameters, so ignore them all.
Also, they add new parameters they find on the end of the list, so the only way to check is to click through to the end of the list.
Anyone notice a drop in Bing traffic on January 26th?
Slashed by 50% and holding for the past three weeks.
100% white hat site
Google used to care if you told them about garbage sites.
I never used that against my competitors, but only on blatant spam,
then just over 2 years ago, google stopped listening to my reports.
The spam don't only stay there, it got worse.
After 3 months of them ignoring every request, I gave up reporting them.
no change that I can tell yet
Thank you algri.
I wouldn't do that.
I think I did something like that a long time ago, but "natural" is a much better thing to do now.
That is what google might call "doing something for the search engines,"
rather than "doing something for your readers."
If your page is really about that keyword, then I believe you would be better off getting that keyword into your page headline, which would be in the H1.
What you are doing is manipulating the power of the H1 to achieve your SEO goal, whereas you should be doing it for your reader, but structuring it so it works for the search engine as well.
I would change what you have.
Having H1 after H2 seems a little strange.
Are you doing that for programming purposes?
Is the H1 or the H2 the same as Title ?
It would seem more "natural" to have H1 first , followed by H2
The SEOMoz crawler ignores robots.txt
It also "indexes" pages marked as noindex.
That means it is filling up the reports with things that don't matter.
Is there any way to stop it doing that?
Whatever you do, you should test and measure.
Only then will you know if it makes a difference.
There don't seem to be any definitive answers.
After you do that, you could tell others the results of your experiments.
Have you tried doing internal URL mapping, rather than allowing two different URLs?
Do you have access to .htaccess?
First detect the mobile device and then do the internal map for that.
If you can do it, there is still only the friendly URL showing externally.
Of course, it depends how the internal system works and how pages are output, but you would just stop the search engines from accessing the internally mapped links.
One reason to change all URLs from _ to - is conformity.
If you have some that are _ and some that are -
the question is how will you remember which one to use, for a particular page?
For that reason, I would convert them all.
As for using a canonical tag, I don't know, you'd need to know what google and otehr search engines do with that information, if anything. I would also worry about what they will do with it in the future, because these things are liable to change.
If it was me, I would change them all and redirect the stragglers.
Yes, use the robots.txt file.
I had my email a friend links in the robots file, then I saw a post here that suggested using robots was bad and that a META noindex,nofollow was better, so I changed to that and it was a disaster.
Google started fetching and indexing those pages. They were even ranking above our content.
And they were forcing our content into supplementary results.
I changed it back to robots, and that fixed the problem, but I think it took a week to undo the damage.
You can use a wildcard asterisk in the robots file, but I don't recommend using more than one wildcard per line.
Disallow: /emailafriend/
Valery's idea sounds better for you.
Rather than ratings, get testimonials and make them rank by adding a link to the testimonials page from the front page of the site, get it into some bookmark sites, add a facebook Like button, maybe a G+ button and get links from a few other relevant places. Be sure to set up the page title, description and content correctly.
The best part of this solution is that it is much easier to get set up and working - you probably already have some good testimonials, and if you don't, then use Steve's idea of following up and asking for a testimonial. If you have a free report to offer them, that will be good, otherwise, just ask for their help.
The followup is a great idea.
It can be next to impossible to get people to rate - or even like - something
If you can make it worth their time, it is more achievable, but still not easy
You probably need to edit that and put it in a code block so it is readable
I would also use a 302.
As long as it doesn't last too long, you should be OK.
If you think it may last longer, you could change your DNS to point to the other location, but you should drop your TTL on DNS so it is a fast changeover to move it back.
This kind of thing is best set up and tested when there are no problems, on another domain, and then test it for real sometime when you have worked out exactly what to do.
If that is causing an infinite loop, then you may have some other code working against the redirect you set up.
Forget page rank on a page - it tells you nothing.
The page is on Huffington Post - It has AOL propping it up and lots of other heavy domains linking to it.
If the page is old and is no longer linked from the front page or immediate sub-pages, it may be possible to beat it. It depends how far down you are and many more things.
It depends what keywords you are going after, so you could use the keyword analysis tool in the Research Tools section. That will give you an idea of how difficult it could be.
You can also use the SEOMoz toolbar and that will tell you how many domains and links are pointing to that page.
Rasmus,
Your description and keywords tags are exactly the same on both pages.
The description should be a summary of the content of each individual page
If you fix that, it may go some way towards fixing the problem.
If that doesn't help, then consider adding more text to those stories
Another thing you could consider is using a different template for that type of page, so you don't add all that stuff under the HR tag, and maybe drop off something from the right column, so the page is a little bit shorter and the story will be a larger percentage of the text.
You also have a lot of inline javascript and a lot of external javascripts.
I don't know if the bots or the ranking algorithms care, but it would be a lot nicer if you consolidated some of that.
Try this, for linux, Tyler:
in .htaccess,
RewriteEngine on
Rewriterule ^/$ http://domain.com/ [R=301,L]
OK thank you, Ralph
I can work on that.
It looks like you have a number of duplicate pages.
Did you just change your sitemap?
Your internal pages are all marked as 0% priority.
That is a weird setting
Good idea, but the problem is the searchers, not the page.
So what I did was add more stuff for those visitors to see and do.
Looks like it is working. Bounce rate is down, time on page is up.
Thank you Ralph.
Yes, had it for years. So is this a GWT report? I thought it was SEOMoz !
No not IIS, Linux.
When I look at the crawl diagnostics, SEOMoz tells me there are 404 errors.
This is understandable, because some pages were removed.
What this report doesn't tell me is how those pages were discovered.
This is a very important piece of information, because it would tell me there are links pointing to those pages, either internal or external. I believe the internal links have been removed.
If the report told me how if found the link, I would be able to take immediate action. Without that information, I have to go so a lot of investigation. And when you have a million pages, that isn't easy.
Some possibilities:
Am I missing something, or is this a feature the SEO Moz crawler doesn't have yet?
What can I do (other than check all my pages) to discover this?
EGOL has a very important answer you need to consider, which is what I was really getting at in my second and last sentences. (but I didn't say it well)
But to answer about the redirect of every page to the root: I don't have any actual experience of this but I wouldn't think it would be a good thing because the front page is untargeted, compared to the page the visitor thought they were going to get.
If doing this was a good idea, there would be no reason for using a 404 page not found. You would instead just redirect all missing pages to the front page.
My guess is that as soon as that pattern was discovered, the value you got from it would evaporate.
Remember that the whole purpose of all those pages was to satisfy the needs of a visitor. Send them to an inappropriate place and you have failed. (by the way, google does this a lot themselves. Their own redirects leave a lot to be desired.)
700 pages is nothing.
It really depends on what you are trying to do, but if you are a programmer, it would be very simple and fast to decide how to split up the files into the final structure, then get a listing of the new locations and generate the .htaccess file from that, because you know the original location.
I would do it by taking an exact copy of the current data and do this in a different location, so as not to cause any disruption. Once it all works, then do a fast swap.
What would make this more complex is if you had to rename the pages, but I would just write a file that contains old,new and use that to generate the needed changes.
interlinking the pages is another job (in their new locations) but if you are a programmer, as noted in the previous line, it is simple.
Something else you must consider is how will the client manage the system afterwards. Be sure not to put them in the situation that they need you in order to add a new page..
At least you are thinking about this before doing it. I just heard from a friend whose client got someone else to restructure his site and they completely destroyed the fantastic #1 and page 1 ranking they spent years creating.