Parse a website using Selenium and Beautifulsoup

PHOTO EMBED

Sun Oct 25 2020 06:36:31 GMT+0000 (Coordinated Universal Time)

Saved by @ianh #python #beautifulsoup

In [8]: from bs4 import BeautifulSoup

In [9]: from selenium import webdriver

In [10]: driver = webdriver.Firefox()

In [11]: driver.get('http://news.ycombinator.com')

In [12]: html = driver.page_source

In [13]: soup = BeautifulSoup(html)

In [14]: for tag in soup.find_all('title'):
   ....:     print tag.text
   ....:     
   ....:     
Hacker News
content_copyCOPY

https://stackoverflow.com/questions/13960326/how-can-i-parse-a-website-using-selenium-and-beautifulsoup-in-python