I'd largely agree with Ron, although there could be a small secondary factor. Many believe that having traffic from a broad range of sources sends a strong quality signal to Google. Links from your emails would count in that respect.

Really though the question is whether adding those links would be useful for the people reading the email.  If they would then add them. Whether or not there is any "SEO benefit" you'll still have happier recipients. Good thing.

If not, then don't bother. If it isn't helpful for them then they won't click the links so no benefit anyway.

It's also worth remembering that most email spam filters look at links in email. Each links will add a little to the likelihood that your email doesn't reach it's intended recipient - so only include them if they're doing a job.

You're welcome.  Let me know if they come back with any questions.

I do this on a few domains and it works well and without a problem.

You should ensure that logged out users (and those who's status you cannot verify) are not directed and then only direct the logged in ones.  That way any crawlers (and logged out users) that come along will not see the redirect at all and just read the url that they requested.

This is slightly different logic to.  It sounds like you might be  testing all users and redirecting them to one of two different URLs - neither of which is the one they requested.  This would be undesirable and would cause the 302 that you describe.

403s are tricky to diagnose because they, by their very nature, don't tell you much.  They're sort of the server equivalent of just shouting "NO!".

You say Moz & Xenu are receiving the 403. I assume that it loads properly from a browser.

I'd start looking at the .htaccess .  Any odd deny statements in there?  It could be that an IP range or user agent is blocked.  Some people like to block common crawlers (Not calling Roger names there).  Check the robots.txt whilst you are there, although that shouldn't return a 403 really.

If you ensure that those pages return a 404 (Not found) response then Google will remove them from it's index in time.

If you want to speed the process up then you can also do this from webmaster tools. There is a guide to removing URLs here: https://support.google.com/webmasters/answer/1663419

It has been said that using the URL removal tools on a lot (read hundreds) of individual URLs might be seen as negative. However if you just have a handful then that would be the way to do it.

The 404s in webmaster tools relate to crawl errors. As such they will only appear if internally linked. It also limits the report to the top 1000 pages with errors only.

I wouldn't try to manually remove that number of URLs.  Mass individual removals can cause their own problems.

If the pages are 404ing correctly, then they will be removed. However it is a slow process. For the number you are looking at it will mostly likely take months.  Google has to recrawl all of the URLs before it even knows that they are returning a 404 status.  It will then likely wait a while and do it again before removing then.  That's a painful truth and there really is not anything much you can do about it.

It might (and this is very arguable) be worth ensuring that there is a crawl path to the 404 content. So maybe a link from a high authority page to a "recently removed content" list that contains links to a selection and keep replacing that list. This will help that content get recrawled more quickly, but it will also mean that you are linking to 404 pages which might send quality signal issues.  Something to weigh up.

What would work more quickly is to mass remove in particular directories (if you are lucky enough that some of your content fits that pattern).  If you have a lot of urls in mysite.com/olddirectory and there is definitely nothing you want to keep in that directory  then you can lose big swathes of URLs in one hit - see here:  https://support.google.com/webmasters/answer/1663427?hl=en

Unfortunately that is only good for directories, not wildcards. However it's very helpful when it is an option.

So, how to find those URLs?  (Your original question!!).

Unfortunately there is no way to get them all back from google.  Even if you did a search for site:www.mysite.com and saved all of the results it will not return the number of results that you are looking for.

I tend to do this by looking for patterns and removing those to find more patterns.  I'll try to explain:

1. Search for site:www.yoursite.com
2. Scroll down the list until you start seeing a pattern. (eg mysite.com/olddynamicpage-111.php , mysite.com/olddynamicpage-112.php , mysite.com/olddynamicpage-185.php  etc) .
3. Note that pattern (return later to check that they all return a 404 )
4. Now search again with that pattern removed, site:www.mysite.com -inurl:olddynamicpage
5. Return to step 2

Do this (a lot) and you start understanding the pattern that have been picked up.  There are usually a few that account for large number of the incorrectly indexed URLs. In the recent problem I did they were almost all relating to "faceted search gone wrong".

Once you know the patterns you can check that the correct headers are being returned so that they start dropping out of the index. If any are directory patterns then you can remove than in big hits through GWMT.

It's painful. It's slow, but it does work.

OK - that is a bit of a different problem (and a rather familiar one).  So the aim is to figure out what the 330 "phantom" pages are and then how to remove them?

Let me know if I have that right. If I have then I'll give you some tips based on me doing to same with a few million URLs recently.  I'll check first though, as it might get long!

As you say, on site crawlers such as Xenu & Screaming frog will only tell you when you are linking to 404 pages, not where people are linking to your 404 pages.

There are a few ways you can get to this data:

Your server logs : All 404 errors will be recorded on your server.  If someone links to a non-existent page and that link is ever followed by a single user or a crawler like google-bot, that will be recorded in your server log files.  You can access those directly (or pull 404s out of them on a regular, automatic basis).  Alternatively most hosting comes with some form of log analysis built in (awstats being one of the most common).  That will show you the 404 errors.

That isn't quite what you asked, as it doesn't mean that they have all been indexed, however that will be an exhaustive list that you can then check against.

Check that backlinks resolve : Download all of your backlinks (OSE, webmaster tools, ahreafs, majestic), look at the target and see what header is returned.  We use a custom build tools called linkwatchman to do this on an automatic regular basis. However as an occasional check you can download in to excel and use the excellent SEO Tools for excel to do this for free. ( http://nielsbosma.se/projects/seotools/  <- best seo tool around)

Analytics : As long as your error pages trigger the google analytics tracking code you can get the data from here as well.  Most helpful when the page either triggers a custom variable, or uses a virtual url ( 404/requestedurl.html for instance).  Isolate the pages and look at where the traffic came from.

Hi SpadaMan,

Submitting articles usually means finding sites that only existing to host other people's articles, giving them a link in exchange for the article content.  That approach has always been flawed and Google has been cracking down on it as deliberate manipulation of the results.  Definitely steer clear.

If your articles are original and useful then your best bet is likely to be to use them on your own website.  Start adding them on a regular basis, and push them out through social media at the same time.  You could contact sites in your niche and see if they would take a and credit you as the author (with a link) as well.

If they aren't original and interesting/useful then your options are going to be limited.  However I'd rather opt for "do nothing" than submitting them to article directories.

Not having the numbers would probably be less confusing than having numbers that might later duplicate.  If you can produce the news sitemaps (and keep them accurate & up to date) the unique numbers are not technically required for Google news listings.  So if they are hard to add and you are confident in your sitemaps it shouldn't be an issue.

If you are less confident in them and being listed on google news is a big deal for you then try to get them added.

that number would work fine... as long as you only ever had 1 news item per day.

That is correct.  We tend to still use a unique number though - partly as a "belt and braces" approach.  The other advantage is that headlines can be updated, keep a headline relevant URL and easily redirect to the correct version.

I had the (mis)fortune of trying to deindex nearly 2 million URLs across a couple of domains recently, so had plenty of time to play with this.

Like CleverPhD I was not able to measure any real difference in the time it took to remove a page that had been 410'd vs one that had been 404'd.

The biggest factor governing the removal of the URLs was getting all the pages recrawled.  Don't underestimate how long that can take.  We ended up creating crawlable routes back to that content to help Google keep visiting those pages and updating the results.