How to find all comments with Beautiful Soup
Two things I needed to do:
First, when importing Beautiful Soup
from bs4 import BeautifulSoup, Comment
Second, here's the code to extract comments
for comments in soup.findAll(text=lambda text:isinstance(text, Comment)):
comments.extract()
You can pass a function to find_all() to help it check whether the string is a Comment.
For example I have below html:
<body>
<!-- Branding and main navigation -->
<div class="Branding">The Science & Safety Behind Your Favorite Products</div>
<div class="l-branding">
<p>Just a brand</p>
</div>
<!-- test comment here -->
<div class="block_content">
<a href="https://www.google.com">Google</a>
</div>
</body>
Code:
from bs4 import BeautifulSoup as BS
from bs4 import Comment
....
soup = BS(html, 'html.parser')
comments = soup.find_all(string=lambda text: isinstance(text, Comment))
for c in comments:
print(c)
print("===========")
c.extract()
the output would be:
Branding and main navigation
============
test comment here
============
BTW, I think the reason why find_all('Comment')
doesn't work is (from BeautifulSoup document):
Pass in a value for name and you’ll tell Beautiful Soup to only consider tags with certain names. Text strings will be ignored, as will tags whose names that don’t match.