python多任务抓取虎牙妹子图片

2018-09-18 06:45:54来源:博客园 阅读 ()

新老客户大回馈,云服务器低至5折

import re
import urllib.request
import gevent


def download(image_download, images_path,i):
    headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"}
    req = urllib.request.Request(url=images_path, headers=headers)
    image = urllib.request.urlopen(req)
    image_content = image.read()

    image_name = image_download + "/" + str(i) + ".jpg"
    file = open(image_name, "wb")
    file.write(image_content)
    file.close()


def main():
    with open("douyu.html", "r", encoding="utf-8") as f:
        messages = f.read()
    images_path = re.findall(r"https://.*?\.jpg", messages)
    directory = "images_douyu"
    i = 0
    gevent_list = list()
    for image_path in images_path:
        i += 1
        gevent_list.append(gevent.spawn(download,directory, image_path, i))

    gevent.joinall(gevent_list)
    

if __name__ == "__main__":
    main()

  

标签:

版权申明:本站文章部分自网络,如有侵权,请联系:west999com@outlook.com
特别注意:本站所有转载文章言论不代表本站观点,本站所提供的摄影照片,插画,设计作品,如需使用,请与原作者联系,版权归原作者所有

上一篇:Python全栈Day 15部分知识点

下一篇:Django框架