web scraping - Getting non-object type randomly when traversing with php DOMDocument -

-

below code:

$xpath = new domxpath($doc); // start root element $query = '//div[contains(@class, "hudpagepad")]/div/ul/li/a'; $nodelist = @$xpath->query($query);  // size 104 $size = $nodelist->length;  ( $i = 1; $i <= $size; $i++ ) {     $node = $nodelist->item($i-1);     $url = $node->getattribute("href");      $error = scrapeurl($url); }  function scrapeurl($url) {     $cfm = new domdocument();     $cfm->loadhtmlfile($url);     $cfmpath = new domxpath($cfm);     $pointer = $cfm->getelementbyid('content-area');     $filter = 'table/tr';      // problem lies here         $state = $pointer->firstchild->nextsibling->nextsibling->nodevalue;      $nodelist = $cfmpath->query($filter, $pointer); }

basically traverses list of links , scrapes each link scrapeurl method.

i don't know problem here randomly non-object type error trying $pointer , passes through without error , values correct.

anyone knows problem here? i'm guessing point when problem occurs when page not loaded properly?

i found idea of answer here:

http://sharovatov.wordpress.com/2009/11/01/php-loadhtmlfile-and-a-html-file-without-doctype/

it better use 'manual' query using getelementbyid coz breaks if doctype of document load not formed.

so use instead:

$cfmpath->query("//*[@id='content-area']")

or create method

function getelementbyid($id) {     global $dom;     $xpath = new domxpath($dom);     return $xpath->query("//*[@id='$id']")->item(0); }

thank attempted help!


Comments

Post a Comment

Popular posts from this blog

django - How can I change user group without delete record -

-
on website user have 1 group. , user can change group. it's made user.groups.clear() and user.groups.add(new_group) but it's not efficient, because there 2 sql query: delete, insert. how can change group update query? user , group related each other using manytomanyfield . means intersection table exists relating both entities, , if don't specify model map (using through attribute) django creates 1 you. looking @ sources django.contrib.auth.models see that's case. fortunatly, can access intermediary model using through attribute of manager (in case, user.groups.through ). can use regular model. example: >>> alice = user.objects.create_user('alice', 'alice@example.com', 'alicepw') >>> employee = group.objects.create(name='employee') >>> manager = group.objects.create(name='manager') >>> alice.groups.add(employee) >>> alice.groups.all() [<group: employee...
Read more

java - Need to add SOAP security token -

-
i need set custom soap header attribute on jax-ws generated webservice client. case webservice calls must go through proxy server requiring specific token (recieved web request header) present in soap request header. e.g.: 1 carserviceservice service = null; 2 service = new carserviceservice(new url(url), new qname(qname); 3 carserviceendpoint port = service.getcarserviceport(); it seems in line 3 wsdl retrieved , call fails due missing security token. 1 point direction on how done? a detailed example has been mentioned here: creating , deploying jax-ws web service on tomcat 6 this article shows how create , use security token .
Read more

java - EclipseLink JPA Object is not a known entity type -

-
i have strange problem eclipselink , object want persist. have 1 object (keypointlistimpl) stores object keypointimpl in list. persisting keypointimpl objects works great if try persist keypointlistimpl object java.lang.illegalargumentexception says object keypointimpl isn't known entity type. here keypointimpl code: @entity @table(name="keypoints") public class keypointimpl implements keypoint { @id @generatedvalue(strategy = generationtype.identity) private int id; @enumerated(enumtype.string) private detectortype keypointtype; private float x; private float y; private float size; private float angle; private float response; private int octave; private int classid; ... } here keypointlistimpl code: @entity @table(name="keypointlists") public class keypointlistimpl implements keypointlist { @id @generatedvalue(strategy = generationtype.identity) private int id; @onetoone(cascade={ca...
Read more