[Python] Fetch texts in all h2 tags with class using BeautifulSoup

Question

I want to extract texts from the H2 tags with class attribute. How can I use BeautifulSoup for it?

Answer 1

You can use the find_all() function to search all the H2 tags on a webpage. In this function, you need to use the parameter "class_" to specify the class value of H2 tags.

In the following code, I search all H2 tags with class="name," then return the text inside the H2 tag.

from bs4 import BeautifulSoup

import urllib.request as ur

url = "url_of_the_webpage"

req = ur.Request(url, None, headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36' })

rs = ur.urlopen(req)

soup = BeautifulSoup(rs, 'html.parser')

for sp in soup.find_all('h2', class_='name'):

        print(sp.text)