How to scroll down with Phantomjs to load dynamic content

Found a way to do it and tried to adapt to your situation. I didn't test the best way of finding the bottom of the page because I had a different context, but check the solution below. The thing here is that you have to wait a little for the page to load and javascript works asynchronously so you have to use setInterval or setTimeout (see) to achieve this.

page.open('http://example.com/?q=houston', function () {

  // Check for the bottom div and scroll down from time to time
  window.setInterval(function() {
      // Check if there is a div with class=".has-more-items" 
      // (not sure if there's a better way of doing this)
      var count = page.content.match(/class=".has-more-items"/g);

      if(count === null) { // Didn't find
        page.evaluate(function() {
          // Scroll to the bottom of page
          window.document.body.scrollTop = document.body.scrollHeight;
        });
      }
      else { // Found
        // Do what you want
        ...
        phantom.exit();
      }
  }, 500); // Number of milliseconds to wait between scrolls

});

I know that it has been answered a long time ago, but I also found a solution to my specific scenario. The result is a piece of javascript that scrolls to the bottom of the page. It is optimized to reduce waiting time.

It is not written for PhantomJS by default, so that will have to be modified. However, for a beginner or someone who doesn't have root access, an Iframe with injected javascript (run Google Chrome with --disable-javascript parameter) is a good alternative method for scraping a smaller set of ajax pages. The main benefit is that it's easily debuggable, because you have a visual overview of what's going on with your scraper.

function ScrollForAjax () {

    scrollintervals = 50;
    scrollmaxtime = 1000;

    if(typeof(scrolltime)=="undefined"){
        scrolltime = 0;
    }

    scrolldocheight1 = $(iframeselector).contents().find("body").height();

    $("body").scrollTop(scrolldocheight1);
    setTimeout(function(){

        scrolldocheight2 = $("body").height();

        if(scrolltime===scrollmaxtime || scrolltime>scrollmaxtime){
            scrolltime = 0;
            $("body").scrollTop(0);
            ScrapeCurrentPage(iframeselector);
        }

        else if(scrolldocheight2>scrolldocheight1){
            scrolltime = 0;
            ScrollForAjax (iframeselector);
        }

        else if(scrolldocheight1>=scrolldocheight2){
            ScrollForAjax (iframeselector);
        }

    },scrollintervals);

    scrolltime += scrollintervals;
}

scrollmaxtime is a timeout variable. Hope this is useful to someone :)