How to get all links from the DOM?
The page.$$eval()
method runs Array.from(document.querySelectorAll(selector))
within the page and passes it as the first argument to the page function.
Since a
in your example represents an array, you will either need to specify which element of the array you want to obtain the href
from, or you will need to map
all of the href
attributes to an array.
page.$$eval()
const hrefs = await page.$$eval('a', links => links.map(a => a.href));
Alternatively, you can also use page.evaluate()
or a combination of page.$$()
, elementHandle.getProperty()
, or jsHandle.jsonValue()
to achieve an array of all links from the page.
page.evaluate()
const hrefs = await page.evaluate(() => {
return Array.from(document.getElementsByTagName('a'), a => a.href);
});
page.$$() / elementHandle.getProperty() / jsHandle.jsonValue()
const hrefs = await Promise.all((await page.$$('a')).map(async a => {
return await (await a.getProperty('href')).jsonValue();
}));
In your example code you're using page.$eval
, not page.$$eval
. Since the former uses document.querySelector
instead of document.querySelectorAll
, the behaviour you describe is the expected one.
Also, you should change your pageFunction
in the $$eval
arguments:
const hrefs = await page.$$eval('a', as => as.map(a => a.href));