Welcome to the Q&A Forum

I've created a robots.txt file for a new Magento install and used an existing site-map that was on the Magento help forums but the trouble is I can't decipher something. It seems that I am allowing and disallowing access to the same expression for pagination. My robots.txt file (and a lot of other Magento site-maps it seems) includes both:

Allow: /*?p=

and

Disallow: /?p=&

I've searched for help on RegEx and I can't see what "&" does but it seems to me that I'm allowing crawler access to all pagination URLs, but then possibly disallowing access to all pagination URLs that include anything other than just the page number?

I've looked at several resources and there is practically no reference to what "&" does...

Can anyone shed any light on this, to ensure I am allowing suitable access to a shop?

Thanks in advance for any assistance

I would never hide H1 tags. I personally would start thinking about completely different templates for mobile and desktop devices and optimise each for their intended target rather than try and serve the same site to both and compromise each user's experience.

From experience, I would highly recommend combining multiple colours onto one page and having an option to select colour, and update photos if required.

We have spent the best part of a year combining variations of products to reduce product pages from 18,000+ to under 1000 and have seen very good SERP improvements because more of the site is being indexed, and there is less duplicate content.

At the end of the day, if nothing changes about a product other than its colour, or size etc then it does not need its own page - it doesn't benefit a user to have 2 pages to browse to make their colour selection.

When you search you might be seeing personalised results. If you search using incognito mode in your browser, and ensure you are not signed into any Google site you should see non-personalised results.

Alternatively, you might have set SEOMoz to check Google from another country, so it's worth checking the campaign settings.

From experience, I would highly recommend combining multiple colours onto one page and having an option to select colour, and update photos if required.

We have spent the best part of a year combining variations of products to reduce product pages from 18,000+ to under 1000 and have seen very good SERP improvements because more of the site is being indexed, and there is less duplicate content.

At the end of the day, if nothing changes about a product other than its colour, or size etc then it does not need its own page - it doesn't benefit a user to have 2 pages to browse to make their colour selection.

When you search you might be seeing personalised results. If you search using incognito mode in your browser, and ensure you are not signed into any Google site you should see non-personalised results.

Alternatively, you might have set SEOMoz to check Google from another country, so it's worth checking the campaign settings.

I would never hide H1 tags. I personally would start thinking about completely different templates for mobile and desktop devices and optimise each for their intended target rather than try and serve the same site to both and compromise each user's experience.