python crawling 삽질3

in python •  6 years ago  (edited)

city = driver.find_element_by_id('shSido')
for option in city.find_elements_by_tag_name('option'):
    if option.text == ' 서울특별시':
        option.click()
driver.find_element_by_xpath(' // *[ @ id = "searchVO"] / div[1] / div / select / option[5]').click()
driver.find_element_by_xpath('//*[@id="icon_btn_write"]').click()
html = driver.page_source
soup = BeautifulSoup(html, 'html.parser')

list = []
try:
while len(list) < 24492:
for i in range(1, 51):
path = '//[@id="searchVO"]/div[2]/table/tbody/tr'
num = i
num = str(num)
xpath = path + '[' + num + ']'
table_list = driver.find_element_by_xpath(xpath).text
test = table_list
list.append(test)
print(list)
driver.find_element_by_xpath('//
[@id="searchVO"]/div[2]/div[2]/ol/li[12]/a').click()
if len(list) == 24492:
print(list)
except:
driver.quit()
pass

'다음'버튼을 누르면 자동으로 페이지가 넘어가게끔 해놔서 에러 남. 페이지 버튼별로 코드를 짜야됨.

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Congratulations @mimiming! You received a personal award!

Happy Birthday! - You are on the Steem blockchain for 1 year!

Click here to view your Board

Support SteemitBoard's project! Vote for its witness and get one more award!

Congratulations @mimiming! You received a personal award!

Happy Birthday! - You are on the Steem blockchain for 2 years!

You can view your badges on your Steem Board and compare to others on the Steem Ranking

Vote for @Steemitboard as a witness to get one more award and increased upvotes!