Why does Linkscap API request hang while extracting data ?

[Ravi_Pathak]

Hi,

I am using LinkScape API to get follow and nofollow links .

I use cron to get data for each url of sitemap.xml.

However while cron is running, the extraction of data hangs on some pages which i later need to delete manually for re starting the execution.

Do anyone have any idea why this is happening ? How can i ignore such pages ?

[Ravi_Pathak]

Hi...

Thank you so much for your reply..

I was implementing  the changeswhich may work.. but they didn't.

I recently came to know about updates to seomoz api which are like,

• SEOmoz Free API: 1 request for every 10 seconds
• Free Extended and PRO Members: 1 request every 5 seconds
• SEOmoz Site Intelligence API: 1 request every 2 seconds

I think this will help me a lot .. thank you ..

[carinoverturf]

Hey Ravi,

Sorry for the delayed response - I wanted to follow up with the engineers to see if they had any suggestions for you.

They agreed the Limit parameter set to 1,000 might be too large to process. Have you tried adjusting that to 300 or even 500? Do you see better success at a lower limit?

Our system will timeout at about 60 seconds so I'm not sure if the hanging is on our end. If dropping the limit size doesn't help, you might want to think about ending the request after about a minute. Sometimes requests that are too long wil timeout, but work fine on a retry as some data will be cached from the previous request.

I hope this information is helpful, but let me know if you're still experiencing issues!

Thanks,
Carin

[Ravi_Pathak]

Some pages are the pages which are moved or not found. I am trying to ignore such pages by getting http response code. But this process takes large amount of time. Almost 10 minutes for a single url of sitemap.xml.

The url of call is like : (php)

http://lsapi.seomoz.com/linkscape/links/URL HERE?SourceCols=4&TargetCols=4&Scope=page_to_page&Sort=page_authority&Limit=1000&Filter=follow&AccessID=" . $accessID . "&Expires=" . $expires . "&Signature=" . $urlSafeSignature;

Thanks .

[carinoverturf]

Hey! This is an issue I haven't heard of before - would you be able to provide anymore information like an example query to the API and some of the pages you are seeing hang?

Thanks!

Carin