Scraping text in h3 and div tags using beautifulSoup, Python
You can use CSS selectors to find the data you need.
In your case div > h3 ~ div
will find all div
elements that are directly inside a div
element and are proceeded by a h3
element.
import bs4
page= """
<div class="box effect">
<div class="row">
<div class="col-lg-10">
<h3>HEADING</h3>
<div><i class="fa user"></i> NAME</div>
<div><i class="fa phone"></i> MOBILE</div>
<div><i class="fa mobile-phone fa-2"></i> NUMBER</div>
<div><i class="fa address"></i> XYZ_ADDRESS</div>
</div>
</div>
</div>
"""
soup = bs4.BeautifulSoup(page, 'lxml')
# find all div elements that are inside a div element
# and are proceeded by an h3 element
selector = 'div > h3 ~ div'
# find elements that contain the data we want
found = soup.select(selector)
# Extract data from the found elements
data = [x.text.split(';')[-1].strip() for x in found]
for x in data:
print(x)
Edit: To scrape the text in heading..
heading = soup.find('h3')
heading_data = heading.text
print(heading_data)
Edit: Or you can get the heading and other div elements at once by using a selector like this: div.col-lg-10 > *
. This finds all elements inside a div
element that belongs to col-lg-10
class.
soup = bs4.BeautifulSoup(page, 'lxml')
# find all elements inside a div element of class col-lg-10
selector = 'div.col-lg-10 > *'
# find elements that contain the data we want
found = soup.select(selector)
# Extract data from the found elements
data = [x.text.split(';')[-1].strip() for x in found]
for x in data:
print(x)