python-xpath语法-爬取彼岸图4k高清动漫壁纸

安装

pip install lxml
1

导入

from lxml import etree
1

xpath使用路径表达式提取html文档中的元素或元素集，然后元素通过沿路径path或步steps来选取数据

XPath常用语法格式

使用xpath匹配数据实践

爬取彼岸图4k高清动漫壁纸 https://pic.netbian.com/4kdongman/

爬取第一页的图片

import requests
from lxml import etree
import os

url = 'https://pic.netbian.com/4kdongman/index.html'
r = requests.get(url)
r.encoding='gbk'
html = etree.HTML(r.text)# 
img_urls = html.xpath("//div[@class='slist']/ul/li/a/@href")# ['/tupian/32274.html', '/tupian/32257.html', ...
for img_url in img_urls:
    # 第二层url
    img_url = 'https://pic.netbian.com' + img_url
    rr = requests.get(url=img_url)
    rr.encoding='gbk'
    img_html = etree.HTML(rr.text)
    img_name = img_html.xpath("//a[@id='img']/img/@title")[0]
    # 高清图片的src
    img_src = 'https://pic.netbian.com' + img_html.xpath("//a[@id='img']/img/@src")[0]
    rimg = requests.get(url = img_src)
    # 可以改文件夹的名字
    folder_name = 'dongman'
    if not os.path.exists(folder_name):
        os.mkdir(folder_name)
    # 保存图片
    with open(f'{folder_name}/{img_name}.jpg','wb') as f:
        f.write(rimg.content)
    print(img_name)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

批量爬取多页图片