UnhandledPromiseRejectionWarning: Error: Page crashed! While using puppeteer

You are assuming, that the error happened because the page navigation failed. This might be the case, but it could also be a different error like a Protocol error. In a case like that, you cannot just reuse the page object, but you first have to restart the browser.

Some time ago, I crawled roughly 400k pages for testing purposes. In total, I experienced 34 of these puppeteer crashes, where some unexpected error crashes the whole browser. To harden your code against these kind of crashes, you need a robust way to restart the browser.

Code Sample

let browser = await puppeteer.launch(/* .. */);
let page = await browser.newPage();

while(true) {
    try {
        // your code that might crash
    } catch (err) {
        try {
            await page.reload(); // soft fix
        } catch (recoveringErr) {
            // unable to reload the page, hard fix
            try {
                await browser.close();
            } catch (err) { // browser close was not necessary
                // you might want to log this error
            }
            browser = await puppeteer.launch(/* .. */);
            page = await browser.newPage();
        }
    }
}

Although this code might look horrendous with three nested try..catch blocks, it does a good job of keeping your code running.

First, your original code is executed. If the initial problem happens, a page.reload call is tried to fix the problem. In case this works, the loop will continue running. If that does not work, the browser will be restarted.

For restarting the browser, I recommend to first try to close the old browser. Although, this might likely fail, it clears all memory and correctly disposes the browser object. Depending on your use case, you might want to log the error or simply ignore it. After disposing the old browser object, you can restart the browser and the loop may continue.

Alternative

As an alternative, you could use my library puppeteer-cluster which has error handling build in. The library automatically restarts the browser in cases in which the page is not usable anymore. In addition, the library can run multiple pages in parallel (as you are trying to stress-test the server I'm assuming).

As per the official documentation, an 'error' event is emitted when the page crashes which can be used to do certain stuff basis the application.

page.on('error', err => { /*custom logic to handle the crash*/ });

For the above specific use case, you could go about doing something like this:

let browser = await puppeteer.launch();
let page = await getNewPage(browser);

while(true){
        try{
            if (page.isClosed()) {
              page = await getNewPage(browser);
            }
            await page.waitFor(selector)
            await page.click(selector)    
        }
        catch(error){
            console.log("FE error with\n\n" + error + "\n\nRefreshing page and continuing profile switching")
            await page.reload();
            continue;
        }}


async function getNewPage(browser) {
    let page = await browser.newPage();
    page.on('error', err => {
        if (!page.isClosed()) {
            //Close page if not closed already
            page.close();
        }
    }
    return page;
}

Reference: page.on('error')