python爬取彼岸图网图片，涉及知识点：requests,xpath,urllib,文件下载后保存，if__name__的用法
网页源码：
<!doctype html>
<html>
<head>
<meta charset="gbk" />
<title>4K壁纸_4K手机壁纸_4K高清壁纸大全_电脑壁纸_4K,5K,6K,7K,8K壁纸图片素材_彼岸图网</title>
<meta name="keywords" content="4K壁纸,4K手机壁纸,4K电脑壁纸,高清壁纸,超清壁纸" />
<meta name="description" content="彼岸图网提供最新最全4K壁纸,4K高清壁纸,4K手机壁纸,4K,5K,6K,7K,8K壁纸,高清图片素材,包含4K游戏、动漫、美女、风景、影视、汽车、动物、人物、美食、背景、平板等精选高清4K壁纸大全" />
<link href="/static/css/style.css" rel="stylesheet" type="text/css" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1">
<script type="text/javascript" src="/static/js/jquery.min.js"></script>
<script type="text/javascript" src="/static/js/common.js"></script>
<script src="/static/js/jquery.SuperSlide.js"></script>
<meta property="qc:admins" content="51744610136654211663757" />
<link rel="icon" href="/favicon.ico" type="image/x-icon"/>
</head>
<body>
<div class="header"><div class="head"><a href="/" class="logo" title="4K壁纸">首页</a><ul class="nav">
<li><a href="/new/" class="nav-link" rel="nofollow">最新</a></li><li> <a href="javascript:;" class="nav-link">分类</a><div class="nav-m clearfix tran"><a href="/4Kxinnian/" title="4K新年图片">4K新年</a><a href="/4kyouxi/" title="4K游戏图片">4K游戏</a><a href="/4kdongman/" title="4K动漫图片">4K动漫</a><a href="/4kmeinv/" title="4K美女图片">4K美女</a><a href="/4kfengjing/" title="4K风景图片">4K风景</a><a href="/4kyingshi/" title="4K影视图片">4K影视</a><a href="/4kqiche/" title="4K汽车图片">4K汽车</a><a href="/4kdongwu/" title="4K动物图片">4K动物</a><a href="/4kbeijing/" title="4K背景图片">4K背景</a><a href="/pingban/" title="平板壁纸图片">平板壁纸</a><a href="/4Kdujia/" title="4K独家图片">4K独家</a><a href="/shoujibizhi/" title="4K手机壁纸图片">4K手机壁纸</a></div></li><li><a href="javascript:;" class="nav-link">尺寸</a><div class="nav-m clearfix tran"> <a href="https://pic.netbian.com/e/search/result/?searchid=1224">4K壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=1224">3840x2160壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=2446">带鱼屏壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=2446">3440x1440壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=2453">5K壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=2453">5120x2880壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=2543">8K壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=2543">7680x4320壁纸</a><a href="/pingban/">平板壁纸</a><a href="https://pic.netbian.com/e/search/result/?searchid=12689" target="_blank">5120x1440壁纸</a><a href="https://pic.netbian.com/shoujibizhi/">4k手机壁纸</a><a href="http://m.netbian.com/" target="_blank">彼岸手机壁纸</a></div></li><li><a href="/4kdongman/" class="nav-link pc-click">4K动漫</a></li><li><a href="/e/search/result/?searchid=2446" class="nav-link pc-click">带鱼屏</a></li><li><a href="/vip.html" class="nav-link" rel="nofollow" style="color:#c39b5e; target=_blank"><b>1元会员</b></a></li>
</ul>
<div class="search"><form action="/e/search/index.php" method="post" name="schform" id="schform"><p><input type="text" name="keyboard" value="请输入壁纸名称" onFocus="if(this.value=='请输入壁纸名称') this.value='';" onBlur="if(this.value=='') this.value='请输入壁纸名称';" autocomplete="off"/></p><input name="submit" type="submit" value="" class="sub" /><input name="tempid" type="hidden" value="1" /><input name="tbname" type="hidden" value="photo" /><input name="show" type="hidden" value="title" /></form></div>
<div class="loginbar"><script>document.write('<script src="/e/member/login/loginjs.php?t='+Math.random()+'"><'+'/script>');</script></div>
</div>
</div>
<div class="wrap clearfix">
<div class="kthylc"><span style="font-size:20px;"><b>声明：注意！目前出现很多复制站，有的连域名都极其相似。本站未发展新站，须细心留意辨别。同时，在这里感谢大家的支持！</b></span></div>
<div id="main"><div class="slide"><ul class="pic">
<li><a href="/vip.html" title="1" target="_blank"><img _src="/d/file/2021/04/30/221552RfgA8.jpg" alt="1" /></a></li><li><a href="https://www.ztupic.com/?sem=bian" title="4" target="_blank"><img _src="/d/file/2024/01/05/175314rDiDw.jpg" alt="4" /></a></li></ul><a class="prev" href="javascript:void(0)"></a><a class="next" href="javascript:void(0)"></a><ul class="hd"></ul></div><script>/*鼠标移过，左右按钮显示*/jQuery(".slide").hover(function(){ jQuery(this).find(".prev,.next").stop(true,true).fadeTo("show",1)},function(){ jQuery(this).find(".prev,.next").fadeOut() });/*SuperSlide图片切换*/jQuery(".slide").slide({titCell:".hd", mainCell:".pic",effect:"leftLoop", autoPlay:true, delayTime:200, switchLoad:"_src",autoPage:true,interTime:20000}); </script>
<div class="classify clearfix"> <a href="/4Kxinnian/" title="4K新年图片">4K新年</a><a href="/4kyouxi/" title="4K游戏图片">4K游戏</a><a href="/4kdongman/" title="4K动漫图片">4K动漫</a><a href="/4kmeinv/" title="4K美女图片">4K美女</a><a href="/4kfengjing/" title="4K风景图片">4K风景</a><a href="/4kyingshi/" title="4K影视图片">4K影视</a><a href="/4kqiche/" title="4K汽车图片">4K汽车</a><a href="/4kdongwu/" title="4K动物图片">4K动物</a><a href="/4kbeijing/" title="4K背景图片">4K背景</a><a href="/pingban/" title="平板壁纸图片">平板壁纸</a><a href="/4Kdujia/" title="4K独家图片">4K独家</a><a href="/shoujibizhi/" title="4K手机壁纸图片">4K手机壁纸</a></div><div class="slist"><ul class="clearfix"><li><a href="/tupian/33061.html" title="古风动漫美女 剑 女侠 4k壁纸" target="_blank"><span><img src="/uploads/allimg/240110/003909-1704818349afc7.jpg" alt="古风动漫美女 剑 女侠 4k壁纸" /></span><b>古风动漫美女 剑 女侠 4k壁纸</b></a></li><li><a href="/tupian/33062.html" title="游泳池 漂亮可爱美女 泳装 4K动漫壁纸图片" target="_blank"><span><img src="/uploads/allimg/240110/004605-1704818765431f.jpg" alt="游泳池 漂亮可爱美女 泳装 4K动漫壁纸图片" /></span><b>游泳池 漂亮可爱美女 泳装 4K动漫壁纸图片</b></a></li><li><a href="/tupian/33059.html" title="以闪亮之名 美少女 4k游戏壁纸" target="_blank"><span><img src="/uploads/allimg/240109/232709-17048140290838.jpg" alt="以闪亮之名 美少女 4k游戏壁纸" /></span><b>以闪亮之名 美少女 4k游戏壁纸</b></a></li><li><a href="/tupian/33056.html" title="《赛博朋克2077》4k游戏壁纸图片" target="_blank"><span><img src="/uploads/allimg/240109/231746-1704813466393f.jpg" alt="《赛博朋克2077》4k游戏壁纸图片" /></span><b>《赛博朋克2077》4k游戏壁纸图片</b></a></li><li><a href="/tupian/33051.html" title="古风美女长发 头饰 花 4k壁纸" target="_blank"><span><img src="/uploads/allimg/240108/231434-1704726874701d.jpg" alt="古风美女长发 头饰 花 4k壁纸" /></span><b>古风美女长发 头饰 花 4k壁纸</b></a></li><li><a href="/tupian/33052.html" title="冬季 雪暮降临 街道 树 汽车 灯光 动漫风景4K壁纸" target="_blank"><span><img src="/uploads/allimg/240108/233905-17047283452c8c.jpg" alt="冬季 雪暮降临 街道 树 汽车 灯光 动漫风景4K壁纸" /></span><b>冬季 雪暮降临 街道 树 汽车 灯光 动漫风景4K壁纸</b></a></li><li><a href="/tupian/33047.html" title="插图 艺术 超宽 女孩 栏杆 5k动漫壁纸" target="_blank"><span><img src="/uploads/allimg/240108/230316-1704726196569c.jpg" alt="插图 艺术 超宽 女孩 栏杆 5k动漫壁纸" /></span><b>插图 艺术 超宽 女孩 栏杆 5k动漫壁纸</b></a></li><li><a href="/tupian/24695.html" title="下午 趴在桌子的女孩4k动漫壁纸3840x2160" target="_blank"><span><img src="/uploads/allimg/190824/212516-1566653116f355.jpg" alt="下午 趴在桌子的女孩4k动漫壁纸3840x2160" /></span><b>下午 趴在桌子的女孩4k动漫壁纸3840x2160</b></a></li><li><a href="/tupian/33027.html" title="幽径空巷4K动漫壁纸" target="_blank"><span><img src="/uploads/allimg/240105/213112-170446147219eb.jpg" alt="幽径空巷4K动漫壁纸" /></span><b>幽径空巷4K动漫壁纸</b></a></li><li><a href="/tupian/33000.html" title="Cyberpunk 2077 2024 赛博朋克5k壁纸" target="_blank"><span><img src="/uploads/allimg/231231/164726-1704012446a57f.jpg" alt="Cyberpunk 2077 2024 赛博朋克5k壁纸" /></span><b>Cyberpunk 2077 2024 赛博朋克5k壁纸</b></a></li><li><a href="/tupian/33017.html" title="古风新娘红色衣服 武术 功夫美女 4k动漫壁纸" target="_blank"><span><img src="/uploads/allimg/240104/012026-1704302426fc6f.jpg" alt="古风新娘红色衣服 武术 功夫美女 4k动漫壁纸" /></span><b>古风新娘红色衣服 武术 功夫美女 4k动漫壁纸</b></a></li><li><a href="/tupian/32974.html" title="报纸 墙 粉红蓝色长头发少女4k动漫壁纸" target="_blank"><span><img src="/uploads/allimg/231228/004217-1703695337659d.jpg" alt="报纸 墙 粉红蓝色长头发少女4k动漫壁纸" /></span><b>报纸 墙 粉红蓝色长头发少女4k动漫壁纸</b></a></li><li><a href="/tupian/32949.html" title="城市街道动漫女孩 摩托车 赛博朋克风格 4k壁纸" target="_blank"><span><img src="/uploads/allimg/231225/005556-17034369564365.jpg" alt="城市街道动漫女孩 摩托车 赛博朋克风格 4k壁纸" /></span><b>城市街道动漫女孩 摩托车 赛博朋克风格 4k壁纸</b></a></li><li><a href="/tupian/32908.html" title="吹泡泡的女孩坐在小巷台阶上 苔藓 小花 4k动漫壁纸" target="_blank"><span><img src="/uploads/allimg/231220/234218-1703086938ecf7.jpg" alt="吹泡泡的女孩坐在小巷台阶上 苔藓 小花 4k动漫壁纸" /></span><b>吹泡泡的女孩坐在小巷台阶上 苔藓 小花 4k动漫壁纸</b></a></li><li><a href="/tupian/32886.html" title="紫色长卷发大波浪美女 报纸 4k动漫壁纸" target="_blank"><span><img src="/uploads/allimg/231219/012739-170292045970ba.jpg" alt="紫色长卷发大波浪美女 报纸 4k动漫壁纸" /></span><b>紫色长卷发大波浪美女 报纸 4k动漫壁纸</b></a></li><li><a href="/tupian/32895.html" title="解限机 Mecha Break 4k游戏壁纸" target="_blank"><span><img src="/uploads/allimg/231219/200129-17029872895dbd.jpg" alt="解限机 Mecha Break 4k游戏壁纸" /></span><b>解限机 Mecha Break 4k游戏壁纸</b></a></li><li><a href="/tupian/32850.html" title="宣纸国画 紫罗兰永恒花园 赛博color 4k壁纸" target="_blank"><span><img src="/uploads/allimg/231214/004320-170248580040da.jpg" alt="宣纸国画 紫罗兰永恒花园 赛博color 4k壁纸" /></span><b>宣纸国画 紫罗兰永恒花园 赛博color 4k壁纸</b></a></li><li><a href="/tupian/32861.html" title="金克丝 英雄联盟LOL 4k游戏壁纸" target="_blank"><span><img src="/uploads/allimg/231215/115152-17026123124601.jpg" alt="金克丝 英雄联盟LOL 4k游戏壁纸" /></span><b>金克丝 英雄联盟LOL 4k游戏壁纸</b></a></li><li><a href="/tupian/32841.html" title="完美世界 清漪 同人美女4k电脑壁纸" target="_blank"><span><img src="/uploads/allimg/231213/233900-1702481940bb59.jpg" alt="完美世界 清漪 同人美女4k电脑壁纸" /></span><b>完美世界 清漪 同人美女4k电脑壁纸</b></a></li><li><a href="/tupian/33050.html" title="原创 2024诸事顺遂4k壁纸" target="_blank"><span><img src="/uploads/allimg/240108/231023-17047266236ace.jpg" alt="原创 2024诸事顺遂4k壁纸" /></span><b>原创 2024诸事顺遂4k壁纸</b></a></li> </ul></div><div class="page"><b>1</b><a href="/index_2.html">2</a><a href="/index_3.html">3</a><a href="/index_4.html">4</a><a href="/index_5.html">5</a><a href="/index_6.html">6</a><a href="/index_7.html">7</a><a href="/index_8.html">8</a><a href="/index_9.html">9</a><a href="/index_10.html">10</a><span class='slh'>…</span><a href="/index_1051.html">1051</a><a href="/index_2.html" class="prev">下一页></a></div></div> <div class="link"><div class="title">友情链接/广告位投放QQ:55346968</div><p><a href='https://pic.netbian.com/' title='4k壁纸' target=_blank>4k壁纸</a><a href='http://www.netbian.com/' title='壁纸' target=_blank>壁纸</a><a href='http://www.netbian.com/' title='电脑壁纸' target=_blank>电脑壁纸</a><a href='http://www.netbian.com/rili/' title='2024年1月日历桌面壁纸' target=_blank>2024年1月日历桌面壁纸</a><a href='https://www.4kbizhi.com/' title='4k桌面壁纸' target=_blank>4k桌面壁纸</a><a href='https://pic.netbian.com/' title='4k' target=_blank>4k</a><a href='https://588ku.com/' title='图片素材 ' target=_blank>图片素材 </a><a href='http://www.88tph.com/' title='图品汇素材' target=_blank>图品汇素材</a><a href='http://www.51miz.com/' title='觅知网' target=_blank>觅知网</a><a href='https://699pic.com/' title='高清图片' target=_blank>高清图片</a><a href='https://win10.ithome.com/' title='Win10之家' target=_blank>Win10之家</a><a href='https://www.design006.com/' title='享设计' target=_blank>享设计</a><a href='http://www.element3ds.com/' title='微元素' target=_blank>微元素</a><a href='http://huke88.com/' title='虎课网' target=_blank>虎课网</a><a href='http://shida66.com/' title='视达网' target=_blank>视达网</a><a href='http://www.1234wu.com/' title='123网址之家' target=_blank>123网址之家</a><a href='https://rrx.cn/' title='H5页面' target=_blank>H5页面</a><a href='http://www.pptfans.cn/' title='PPT模板' target=_blank>PPT模板</a><a href='http://www.99ppt.com/' title='当图网' target=_blank>当图网</a><a href='http://www.yanj.cn/' title='PPT模板' target=_blank>PPT模板</a><a href='https://www.fotor.com.cn/' title='素材库' target=_blank>素材库</a><a href='http://www.wanmeimv.com/' title='mtv歌曲' target=_blank>mtv歌曲</a><a href='https://www.ifabiao.com/' title='爱发表' target=_blank>爱发表</a><a href='http://www.51yuansu.com' title='觅元素' target=_blank>觅元素</a><a href='https://www.ypppt.com/' title='ppt模板' target=_blank>ppt模板</a><a href='https://www.ppt20.com/' title='变色龙PPT' target=_blank>变色龙PPT</a><a href='http://www.pptjia.com/' title='pppt模板' target=_blank>pppt模板</a><a href='https://www.shen88.cn/' title='十二星座排名' target=_blank>十二星座排名</a><a href='https://www.photophoto.cn/' title='图行天下' target=_blank>图行天下</a><a href='https://www.tusij.com/' title='在线图片制作' target=_blank>在线图片制作</a><a href='http://xsj.699pic.com/' title='高清图片' target=_blank>高清图片</a><a href='https://www.yasuotu.com/' title='图片压缩' target=_blank>图片压缩</a><a href='https://zhutix.com/' title='主题' target=_blank>主题</a><a href='https://www.tusij.com/' title='在线ps' target=_blank>在线ps</a><a href='https://www.bigbigwork.com/' title='大作' target=_blank>大作</a><a href='https://www.win7xzb.com/' title='win7旗舰版' target=_blank>win7旗舰版</a><a href='https://www.vjshi.com/' title='视频素材' target=_blank>视频素材</a><a href='https://www.qqw21.com/' title='头像图片' target=_blank>头像图片</a><a href='https://tianqi.2345.com/' title='天气预报' target=_blank>天气预报</a><a href='https://www.huashi6.com/' title='动漫壁纸' target=_blank>动漫壁纸</a><a href='http://www.333cn.com/' title='设计之窗' target=_blank>设计之窗</a><a href='https://ibaotu.com/' title='包图网' target=_blank>包图网</a><a href='https://www.ztupic.com/' title='图片素材' target=_blank>图片素材</a><a href='https://editor.588ku.com/' title='千库编辑' target=_blank>千库编辑</a> </p></div>
</div>
<div id="footer">
<div class="w"><p>彼岸图网为用户免费分享产生,请勿用于商业用途,如有侵权问题请及时联系我们,我们会尽快处理。客服QQ:55346968 <a target="_blank" href="//wpa.qq.com/msgrd?v=3&uin=55346968&site=qq&menu=yes"><img border="0" src="/static/img/qqonline.jpg" alt="点击这里给我发消息" title="点击这里给我发消息"/></a>（工作时间：9:00-23:00）</p>
<p>Copyright 2016-2024 <span class="sitename"><a href="https://pic.netbian.com/">彼岸图网</a></span> <a href="https://beian.miit.gov.cn/" target="_blank">闽ICP备13013111号-1</a> <a href="https://pic.netbian.com/" target="_blank" title="高清壁纸">高清壁纸</a> <a href="/bangzhu.html" target="_blank">帮助</a> <a href="/tougao.html" target="_blank">壁纸投稿</a>
</p>
</div>
</div>
<div class="gotop">
<div class="bd">
<div class="btn-qq"><i></i>客服<div class="kf-trop kf-qq"><p class="kf-name">在线客服</p><p>工作时间(9:00-23:00)</p><p class="kf-img"><a href="//wpa.qq.com/msgrd?v=3&uin=55346968&site=qq&menu=yes"><img border="0" src="/static/img/qqonline.jpg" alt="点击这里给我发消息" title="点击这里给我发消息"/></a></p><p class="kf-name kf-faq"><a href="/bangzhu.html" target="_blank">常见问题>></a></p></div></div>
<a href="javascript:;" class="btn-btn btn-phone"><i></i>电话<div class="kf-trop"><p>服务热线</p><b>13605998361</b><p>工作时间(9:00-23:00)</p></div></a>
<div class="btn-btn btn-group"><i></i>QQ群<div class="kf-trop"><h4>QQ群</h4><p><span>QQ群4（20379343）</span><a>推荐3</a></p><p><span>QQ群3（223739596）</span><a>推荐2</a></p><p><span>QQ群2（864686752）</span><a>推荐1</a></p><p><span>QQ群1（427389122）</span><a>热门</a></p><div class="more"><a href="">更多</a></div></div></div>
</div>
<a href="javascript:;" class="btn-btn btn-top"><i></i>顶部</a>
</div>
</body>
</html>python代码：
# 保存文件库
import os.path
import random
import timeimport requests
from lxml import etree
from urllib import request as requrl = 'http://pic.netbian.com/'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
response = requests.get(url, headers=headers)
response.encoding = 'GBK'
# print(response.text)
tree = etree.HTML(response.text)
# 获取到了当前所有的图片的li
li = tree.xpath('//ul[@class="clearfix"]/li')
for i in li:# 获去图片urlhref = i.xpath('./a//img/@src')# 拼凑完整urlimage_url = 'http://pic.netbian.com' + href[0]# 获取文件名image_name = i.xpath('./a/b/text()')[0]# print(image_url)# 定义文件路径path = './img'# 如果当前路径不存在 则创建if not os.path.exists(path): # 创建路径os.mkdir(path)# 下载链接和图片保存的位置req.urlretrieve(image_url, os.path.join(path, image_name+'.jpg'))print(image_name)# # 等待间隔时间time.sleep(random.randint(1,3))
# 1 练习 抓取多页
# 2 练习 抓取详情页的大图
# 3 练习 抓取多页 详情页的大图运行结果：
添加if__name__后的写法：
import os.path
import random
import time
import requests
from lxml import etree
from urllib import request as reqdef main(url):headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',}response = requests.get(url, headers=headers)response.encoding = 'GBK'# print(response.text)tree = etree.HTML(response.text)# 获取到了当前所有的图片的lili = tree.xpath('//ul[@class="clearfix"]/li')for i in li:# 后去图片urlhref = i.xpath('./a//img/@src')# 拼凑完整urlimage_url = 'http://pic.netbian.com' + href[0]image_name = i.xpath('./a/b/text()')[0]# print(image_url)path = './img'# 如果当前路径不存在 则创建if not os.path.exists(path):os.mkdir(path)req.urlretrieve(image_url, os.path.join(path, image_name+'.jpg'))print(image_name)time.sleep(random.randint(1,3))if __name__ == '__main__':url = 'http://pic.netbian.com/'main(url)
# 1 练习 抓取多页
# 2 练习 抓取详情页的大图
# 3 练习 抓取多页 详情页的大图