爬个豆瓣图片,记录个小问题,发现爬取豆瓣原图的时候拿不到数据,返回为空,爬小图可以,爬其他网站的也都正常,最后发现是header中If-Modified-Since这个参数的原因,加上了就拿不到数据,去掉就行。
-
- import requests
-
- headers = {
- 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
- 'Accept-Language': 'zh-CN,zh;q=0.9,ru;q=0.8,en;q=0.7,zh-TW;q=0.6',
- 'Cache-Control': 'max-age=0',
- 'Connection': 'keep-alive',
- # 'If-Modified-Since': 'Sun, 03 Nov 2019 04:42:19 GMT',
- 'Referer': 'https://movie.douban.com/celebrity/1010504/photo/2421104812/',
- 'Sec-Fetch-Dest': 'document',
- 'Sec-Fetch-Mode': 'navigate',
- 'Sec-Fetch-Site': 'cross-site',
- 'Sec-Fetch-User': '?1',
- 'Upgrade-Insecure-Requests': '1',
- 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36',
- 'sec-ch-ua': '"Chromium";v="116", "Not)A;Brand";v="24", "Google Chrome";v="116"',
- 'sec-ch-ua-mobile': '?0',
- 'sec-ch-ua-platform': '"Windows"',
- }
- response = requests.get('https://img2.doubanio.com/view/photo/raw/public/p2421104812.jpg', headers=headers)
- content = response.content
- with open('temp.jpg','wb') as f:
- f.write(content)