• Python学习之CSDN21天学习挑战赛计划之14

    ​目录

    1，request模块总结

    2，安装requests

    3，向百度发送请求

    4 发送带headers参数请求

    5，headers参数中携带cookie

    活动地址：CSDN21天学习挑战赛

    学习的最大理由是想摆脱平庸，早一天就多一份人生的精彩；迟一天就多一天平庸的困扰。

    1，request模块总结

    该模块主要用于发送请求获取响应，该模块有很多的替代模块，比如说urtlib模块，工作中用的比较多还是requests模块，requests的代码简洁易懂，相对于臃肿的urlib模块，使用requests编写的爬虫代码将会更少，而且实现某些功能将会简单

    2，安装requests

    1. PS C:\Users\T14\Desktop> pip install requests
    2. Collecting requests
    3.   Downloading requests-2.28.1-py3-none-any.whl (62 kB)
    4.      ---------------------------------------- 62.8/62.8 KB 30.1 kB/s eta 0:00:00
    5. Collecting urllib3<1.27,>=1.21.1
    6.   Downloading urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
    7.      ---------------------------------------- 139.9/139.9 KB 24.3 kB/s eta 0:00:00
    8. Collecting certifi>=2017.4.17
    9.   Downloading certifi-2022.6.15-py3-none-any.whl (160 kB)
    10.      ---------------------------------------- 160.2/160.2 KB 26.3 kB/s eta 0:00:00
    11. Requirement already satisfied: charset-normalizer<3,>=2 in c:\users\t14\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from requests) (2.1.0)
    12. Collecting idna<4,>=2.5
    13.   Downloading idna-3.3-py3-none-any.whl (61 kB)
    14.      ---------------------------------------- 61.2/61.2 KB 19.2 kB/s eta 0:00:00
    15. Installing collected packages: urllib3, idna, certifi, requests
    16. Successfully installed certifi-2022.6.15 idna-3.3 requests-2.28.1 urllib3-1.26.11

    3，向百度发送请求

    1. import requests
    2.  
    3. # 目标网址
    4. url = "http://www.baidu.com/"
    5.  
    6. # 发送请求获取响应
    7. response = requests.get(url)
    8.  
    9. # 查看响应对象的类型
    10. print(type(response))
    11. # 查看响应状态码
    12. print(response.status_code)
    13. # 查看响应内容的类型
    14. print(type(response.text))
    15. # 查看cookies
    16. print(response.cookies)
    17. # 查看响应的内容
    18. print(response.text)

    输出结果如下

    1. ==================== RESTART: C:/Users/T14/Desktop/test15.py ===================
    2. <class 'requests.models.Response'>
    3. 200
    4. <class 'str'>
    5. <Cookie BDORZ=27315 for .baidu.com/>]>
    6. html>
    7.  
    8. <html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=http://s1.bdstatic.com/r/www/cache/bdorz/baidu.min.css><title>ç¾åº¦ä¸ä¸ï¼ä½ å°±ç¥étitle>head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus>span><span class="bg s_btn_wr"><input type=submit id=su value=ç¾åº¦ä¸ä¸ class="bg s_btn">span> form> div> div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>æ°é»a> <a href=http://www.hao123.com name=tj_trhao123 class=mnav>hao123a> <a href=http://map.baidu.com name=tj_trmap class=mnav>å°å¾a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>è§é¢a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>è´´å§a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>ç»å½a> noscript> <script>document.write('encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">ç»å½');script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">æ´å¤äº§åa> div> div> div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>å³äºç¾åº¦a> <a href=http://ir.baidu.com>About Baidua> p> <p id=cp>©2017 Baidu <a href=http://www.baidu.com/duty/>使ç¨ç¾åº¦åå¿è¯»a>  <a href=http://jianyi.baidu.com/ class=cp-feedback>æè§åé¦a> äº¬ICPè¯030173å·  <img src=//www.baidu.com/img/gs.gif> p> div> div> div> body> html>
    9.  
    10.  
    11. >>>

    4 发送带headers参数请求

    1. import requests
    2.  
    3. # 目标网址
    4. url = "http://www.baidu.com/"
    5.  
    6. # 构建请求头字典，最重要的就是User-Agent
    7. # 如果需要其他请求头，就在headers字典中加上
    8. headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36'}
    9.  
    10. # 发送请求获取响应
    11. response = requests.get(url,headers=headers)
    12.  
    13. print(response.text)

    输出结果太多了，就不贴了。。。

    5，headers参数中携带cookie

    添加Cookie可以模拟普通用户的请求，由于有时效性，因此过段时间需要更换

  • 相关阅读:
    微同城生活圈小程序源码系统 专业搭建本地生活服务的平台 带完整搭建教程
    Audacity开源音频处理软件使用入门
    003-JAVA运行带参数的程序，用Scanner类获取键盘输入
    uni-app - 日期 · 时间选择器
    图像分割简述
    【Python】运算符
    用pyinstaller打包LGBM模型为ELF/EXE可执行文件
    npm版本错误——npm ERR! code ERESOLVE 解决方法
    Hadoop和Spark的对比
    zookeeper异常 Cannot open channel to 3 at election address
  • 原文地址：https://blog.csdn.net/Cupid99/article/details/126373806