and you don't need to start with it : Start with smart, or with wizard, and then Edit in Advanced Mode. but Octoparse tries to do it for you.īut of course, the Advanced Mode is the most important part. Sometimes you need to find alternate ones. Smart Mode and Wizard mode make it easy to find the data, often at first sight. hidden behind an 'Display' Ajax button that I wasn't able to deal with (with php / cUrl)ġ0 tasks are offered for free, and as far I know, won't be public tasks as it's the case with some of Octoparse competitors because I was unable to access the most important part of the data I needed. as if it wouldn't be any ajax routines on the pages. I gave a try to some scraping tools, and my final choice was made to Octoparse.Ījax is handled as easy as a basic html url. So, I had to find a way to still be able to extract my needed data, without having to pass an engineer degree in information technology.
and the dynamic pages that don't load at first sight, that wait for you to click on a button, that just show as you scroll down, that exchange static pictures urls with javascipt dynamically shown pictures. Then came for me (and I must admit, my limited skills) THE hammer : AJAX ! Yes, html + Javascipt + css + dom. In fact, websites regularly change minor things on their pages, and in the best case, you wouldn't get anymore some or all of the awaited data, in the worse case, absolutely inaccurate data. Years after years, it sounded clear that my extracting routines running on my server were more and more difficult to maintain in a good working shape. I have been crawling and parsing websites for a while, with use of php and cUrl.
0 kommentar(er)
