首页 > > 网络编程 > 其它 >

scrapy爬取校花网男神图片保存到本地

2018-10-15 02:29:42来源：博客园阅读 ()

爬虫四部曲，本人按自己的步骤来写，可能有很多漏洞，望各位大神指点指点

1、创建项目

scrapy startproject xiaohuawang

scrapy.cfg: 项目的配置文件
xiaohuawang/: 该项目的python模块。之后您将在此加入代码。
xiaohuawang/items.py: 项目中的item文件.
xiaohuawang/pipelines.py: 项目中的pipelines文件.
xiaohuawang/settings.py: 项目的设置文件.

2、进入xiaohuawang目录然创建爬虫名及爬取范围

cd xiaohuawang

scrapy genspider xhwang xiaohuar.com

3、创建爬取的项目（xiaohuawang/item.py）

4、编写爬虫

xiaohuawang/spider/xhwang.py: 爬虫文件

5、保存数据（此处使用try...expect是为剔除完整的图片url）

6、打开数据保存开关（setting.py）如果有第五步的保存数据的必要,此处需要打开

7、开始爬取之路

在终端键入scrapy crawl xhwang 本地目录出现如下数据（截取部分）校花还看得过去，这校草嘛除了李易峰其他都没见过o(*￣︶￣*)o

终端出现的界面

2018-10-15 09:52:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/7e191d68e7eb029b4c1da1a3808e5059.jpg HTTP/1.1" 200 53517
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/63d5b955486e38d0fb664bcc9e8d96d6.jpg HTTP/1.1" 200 32479
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:26 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20160109/1b82558a72a175802de1c041d7c60813.jpg HTTP/1.1" 200 779046
2018-10-15 09:52:31 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:31 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170608/f07501a9d1fe9945e2fb96f0890a8d23.jpg HTTP/1.1" 200 113462
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151018/e03c74b0e1fe13caab5e138c8b013a00.jpg HTTP/1.1" 200 33869
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9c9e3de9965854e46404af656d86a596.jpg HTTP/1.1" 200 212437
2018-10-15 09:52:34 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:35 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/180584665e90af45905e3ab2ab0bdb62.jpg HTTP/1.1" 200 32805
2018-10-15 09:52:35 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:35 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f96fe78647d53691e5cb3d9f483d950e.jpg HTTP/1.1" 200 161345
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/1b3b53eeae45059d3f46490d3eb45672.jpg HTTP/1.1" 200 71261
2018-10-15 09:52:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/8cb2bf048b0fdcbb8aabe6ca599eaa2b.jpg HTTP/1.1" 200 78371
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/d02e95d32d1aed5fb46d5744ed5f9729.jpg HTTP/1.1" 200 14788
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/65d5cfc0d5f54559c07073edf2421da7.jpg HTTP/1.1" 200 836584
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/901b797f3ed0c5e1c7e861496ebe1c2d.jpg HTTP/1.1" 200 75831
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/3047a6706453b987c26f54beba3e32ae.jpg HTTP/1.1" 200 81777
2018-10-15 09:52:44 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:44 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151003/9177cd57017a50783a94a4c700bae808.jpg HTTP/1.1" 200 194601
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/b332da0c2a45f5ec08add990255b0071.jpg HTTP/1.1" 200 31974
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:52:46 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:47 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0540ae30cfbdf80a8121d5d6e14e8b62.jpg HTTP/1.1" 200 80261
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/7e9a5cf7fd86a4afd8da51afe24cf82d.jpg HTTP/1.1" 200 48760
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:48 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/ef1f9ff054246626f7a3edbcaab11810.jpg HTTP/1.1" 200 85468
2018-10-15 09:52:49 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:49 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/67af1100e5a0ba7e0de35f06e6d3c5fe.jpg HTTP/1.1" 200 46750
2018-10-15 09:52:49 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:50 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/4e0406f4073c4120b6e93cb08eafaf76.jpg HTTP/1.1" 200 11429
2018-10-15 09:52:50 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:50 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150915/f51d151afdd59c2f0b94ae5ebd9df2e5.jpg HTTP/1.1" 200 187448
2018-10-15 09:52:51 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:51 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/974244ea878d9d58cdeb5e581ba00c2d.png HTTP/1.1" 200 51702
2018-10-15 09:52:51 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:52 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/4bbf1aebc9e0ff735539e47185f07d86.jpg HTTP/1.1" 200 55832
2018-10-15 09:52:52 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:52 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/79f175dd8866f8bd756d0d663b56f63f.jpg HTTP/1.1" 200 97644
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20160128/edb010401a9314f51ba85dcef5628640.jpg HTTP/1.1" 200 51555
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:53 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/38c8daefae96096777168b878ac12457.jpg HTTP/1.1" 200 48988
2018-10-15 09:52:54 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:54 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f57c0b3d74190fcdf3496f29c495a9a8.jpg HTTP/1.1" 200 89019
2018-10-15 09:52:55 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:55 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20161209/b0629634f424d3fc92ccf5e3311cf26d.jpg HTTP/1.1" 200 233679
2018-10-15 09:52:56 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:56 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0d1c02b6d4eb2d49e06b85c920226e12.jpg HTTP/1.1" 200 35265
2018-10-15 09:52:56 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:57 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150917/1ea043c66124620e7bef39abfeaa7309.jpg HTTP/1.1" 200 102374
2018-10-15 09:52:59 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:52:59 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/ba9f9d31a9c1a74c0992fc53642351b4.jpg HTTP/1.1" 200 48760
2018-10-15 09:53:00 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:00 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/42d1ff3f5ee54c737a2a20691a51dd2e.jpg HTTP/1.1" 200 117388
2018-10-15 09:53:01 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:01 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151004/ff3c6b4a769142bcf07d36a38090e9ca.jpg HTTP/1.1" 200 238026
2018-10-15 09:53:02 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:02 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/54859a36a815d4cc9e4bfd0fa226a801.jpg HTTP/1.1" 200 65610
2018-10-15 09:53:03 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:03 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150829/61d547839957d38499af8a030f181bbc.jpg HTTP/1.1" 200 34660
2018-10-15 09:53:06 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:06 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170210/8e6c33aec8013aee65eb677f64ead2ff.jpg HTTP/1.1" 200 144376
2018-10-15 09:53:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:07 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/54da0d784895352c52c8de4c4724702b.jpg HTTP/1.1" 200 70823
2018-10-15 09:53:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:08 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151229/f7235e51a566a9b25343a3a034f47291.jpg HTTP/1.1" 200 58763
2018-10-15 09:53:08 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:08 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/ed3640812048582f544140c1dc29c701.jpg HTTP/1.1" 200 69413
2018-10-15 09:53:10 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:10 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/72edbf7ff56c981b82effd1dd2a181ff.jpg HTTP/1.1" 200 101817
2018-10-15 09:53:13 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:13 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9aad0c431d4508d2924364532e5fb405.jpg HTTP/1.1" 200 77562
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/6cc7ad88b47e1191cb112deed98a96cd.jpg HTTP/1.1" 200 21454
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0cb7df1d7ee292ddf6a7de7c9084e97c.jpg HTTP/1.1" 200 96486
2018-10-15 09:53:15 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:15 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170221/a6e051fd0752d6ad3beca2014a1f620e.jpg HTTP/1.1" 200 31762
2018-10-15 09:53:15 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:16 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/895782554cccd62f3819c0f912d8afba.jpg HTTP/1.1" 200 60290
2018-10-15 09:53:16 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:16 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/1a645a5896cc1a2989c9778c0dcd3d4e.jpg HTTP/1.1" 200 36327
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c077e29c5f9b89eb404505567df17c0c.jpg HTTP/1.1" 200 20020
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f825758f36e1c2c346f4f8f957df5430.jpg HTTP/1.1" 200 39516
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:17 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/782cccaab95f8bdbcd6937c67cbaa3b0.jpg HTTP/1.1" 200 380076
2018-10-15 09:53:22 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:22 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/97f350f3e0fe65a8f7c768664e1eaca5.jpg HTTP/1.1" 200 422140
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/883ea459b664518f60d9ea6fd9f49f0d.jpg HTTP/1.1" 200 5545
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/281fd6f85bf33ace9b99b02031cb2cb9.jpg HTTP/1.1" 200 55469
2018-10-15 09:53:26 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:26 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/46cfceb6eac4bd6d5ab051098f413bdf.jpg HTTP/1.1" 200 132728
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c62ebcca53796107aaad2fe5577cde44.jpg HTTP/1.1" 200 10273
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.comhttp:80
jpg_url is error!!!
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:27 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2219bc2968708eb47ca47b618c96e003.jpg HTTP/1.1" 200 128837
2018-10-15 09:53:28 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:28 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/50c4400bda96f2305fc74de79aea913b.jpg HTTP/1.1" 200 42574
2018-10-15 09:53:29 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:29 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/cfdffb8e41273fbc287293bd2d0a3e59.jpg HTTP/1.1" 200 40472
2018-10-15 09:53:30 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:30 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20150905/ffa892119baef28dadfcadc3b7abd50a.jpg HTTP/1.1" 200 128271
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20170331/5164f1dc347b2cd5d6b57088d2614c4d.jpg HTTP/1.1" 200 30121
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:31 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/299823f15c021ede81f844b30b8d543a.jpg HTTP/1.1" 200 76565
2018-10-15 09:53:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/bf464364888cfecaf95b0a2a9e3407bf.png HTTP/1.1" 200 305039
2018-10-15 09:53:36 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:36 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/a9843ed50a56e020ed78e40a348bcb5c.jpg HTTP/1.1" 200 87337
2018-10-15 09:53:37 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:37 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/f9fe702b5e01210af5afd970e8499090.jpg HTTP/1.1" 200 107942
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/d3e3d2526686463b4a1406e039388ca6.jpg HTTP/1.1" 200 58479
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:38 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/1e29bc94ee7ebde0ae50bc2b5bd112c9.jpg HTTP/1.1" 200 66940
2018-10-15 09:53:40 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:40 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/db4d5e02fdea2e910ee35eb6b4979a99.jpg HTTP/1.1" 200 7716
2018-10-15 09:53:40 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:41 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/afaf7502ffca5a0bb052eddba8195509.jpg HTTP/1.1" 200 47045
2018-10-15 09:53:41 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:41 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/90d63585d7230a48ddd9516dbf0dd6ab.jpg HTTP/1.1" 200 74944
2018-10-15 09:53:42 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/689fa17c65c8d0f049521383d43a4ab2.jpg HTTP/1.1" 200 49714
2018-10-15 09:53:43 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:43 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151217/b5108ea8a7a4920b2d7bf6e27c0b1101.jpg HTTP/1.1" 200 111850
2018-10-15 09:53:44 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:44 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/e81bf99a232cdeee60dc65418a8c2602.png HTTP/1.1" 200 46237
2018-10-15 09:53:45 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:53:45 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/04c92ce23ee91e9cd857955e4a60b46d.jpg HTTP/1.1" 200 994018
2018-10-15 09:54:00 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:00 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9a3315019ae4b15e84e18770f91e7f28.jpg HTTP/1.1" 200 99168
2018-10-15 09:54:02 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:02 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2d64c4f98c23b4177ee1ae9725354ea4.jpg HTTP/1.1" 200 258723
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0ec2c52c6954837c2b588bb61e7e2e15.jpg HTTP/1.1" 200 47123
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:07 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20160220/b1e09b5d13c74f444806086a1a8aedbe.jpg HTTP/1.1" 200 33438
2018-10-15 09:54:08 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:11 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/6854f7987613ceac3a21049efa2f7047.jpg HTTP/1.1" 200 45545
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/873316b54c2bc3bcc3573971fda73286.png HTTP/1.1" 200 36981
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:14 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/117e999a87ef237115071d64b692b134.jpg HTTP/1.1" 200 286144
2018-10-15 09:54:16 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:16 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/24191b9f48d9f0ee58baf69e13a18c78.jpg HTTP/1.1" 200 704182
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2568b473a4a93b0be1366800d9194799.jpg HTTP/1.1" 200 40740
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:21 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/0a83bccd885cba88a11eb93c5f789035.jpg HTTP/1.1" 200 102835
2018-10-15 09:54:22 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:23 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/a55cc2a57a2af7fad3477d799215ad05.jpg HTTP/1.1" 200 63726
2018-10-15 09:54:23 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/cd38314fd9c336afe56ef8d61cb8416d.jpg HTTP/1.1" 200 53809
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c0a77ff668ed2bb6292ec81a4c4105c7.jpg HTTP/1.1" 200 49287
2018-10-15 09:54:24 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/2bb8f36b6f540360d081ea17dad0d1c6.jpg HTTP/1.1" 200 59997
2018-10-15 09:54:25 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:25 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/20151213/e6f03eb1bb25208f230338ee76963d73.jpg HTTP/1.1" 200 126816
2018-10-15 09:54:28 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:28 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/c6bfce8e88fbeeb0ec94ed740ea02657.jpg HTTP/1.1" 200 25190
2018-10-15 09:54:28 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:29 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/9732937ef6b527bc9374bec637f209b4.jpg HTTP/1.1" 200 99894
2018-10-15 09:54:32 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): www.xiaohuar.com:80
2018-10-15 09:54:32 [urllib3.connectionpool] DEBUG: http://www.xiaohuar.com:80 "GET /d/file/e0f286f6c4a83c5458c99e98d692a383.png HTTP/1.1" 200 46237
2018-10-15 09:54:33 [scrapy.core.scraper] DEBUG: Scraped from <200 http://www.xiaohuar.com/xiaocao/>
{'name': ['大连海事大学校草牟福鹏',
'广西农业职业技术学院校草王超树',
'四川师范大学校草李易峰',
'南开中学校草王源',
'重庆八中校草王俊凯',
'清华大学校草林时完',
'北京市海淀外国语实验学校校草鹿晗',
'北京师范大学珠海分校校草吴凡',
'湖南重点中学湖南师大附中校草易烊千玺',
'中国文化大学校草柯震东',
'清华大学校草刘睿睿',
'北京大学校草苑子文苑子豪',
'清华大学校草李润东',
'辽宁石油化工大学顺华能源学院校草李佩凡',
'新疆警官高等专科学校校草杨晨',
'浙江大学校草王平',
'云南国土资源学院校草李莘瞳',
'吉林大学校草夏昕濛',
'中国人民大学校草龙马',
'湖南师范附中梅溪湖中学校草易烊千玺',
'华中科技大学校草杨逸韬',
'四川师范大学校草潘子剑',
'四川警察学院校草孟祥鑫',
'大连理工校草仪炎彬',
'深圳大学校草甘东柏',
'中国民航飞行学院校草木拉丁',
'西南大学校草温浩',
'兰州交通大学校草柴铎',
'北京现代音乐学院校草陈沫燃',
'长沙理工大学城南学院校草黄庭帅',
'福建师范大学校草李晨希',
'中南民族大学校草李春天',
'清华大学校草孟湛东',
'浙江警察学院校草张超贺',
'杭州师范大学校草俊杰',
'白城师范学院校草袭新',
'江苏省司法警官高等职业学校校草魏一川',
'集美大学校草廖家汉',
'华中师范大学校草邹祖樊',
'中国美院校草凤飞飞',
'山东服装职业学院校草徐义强',
'江西南昌安义二中校草校草黄凡斌',
'上海杉达学院校草沈祎玮',
'哈密石油高级中学校草王竞',
'辽宁水利学院校草小马',
'广西贵港屌丝中学校草苏泽辰',
'华南师范大学校草沈斌',
'河南理工校草范博',
'寒校草秋',
'西南财经大学天府学院校草周月笙',
'南昌大学校草邓文才',
'啊实打实的校草阿萨德',
'北京师范大学校草秋寒',
'铁岭市外事学院校草董立鹏',
'四川帅哥校草刘亦晨',
'长春汽车工业高等专科学校校草史培元',
'南昌航空大学校草郭美城',
'云南警官学院校草李庆?',
'安徽财经大学校草查道杰',
'四川师范大学校草刘亦晨',
'赣南师范学院校草王思豪',
'福建师范大学校草李毅刚',
'北京电影学院校草巴宇嘉',
'西南政法大学校草董浩洋',
'安徽大学校草张峰',
'廊坊东方大学城河北体育学院校草馮政',
'珠海大学校草蔡璇',
'江南高中校草朴朗',
'重庆南开中学高中部校草王源',
'长春工程学院校草邸平涛',
'北京八维大学校草李超',
'玉溪校草杨玉金',
'上海师范大学天华学院校草沈辛峄',
'校草校草白天明',
'技师学院校草刘家森',
'泉州二中校草安然',
'专门磕药大学校草丁三圆',
'哈尔滨工业大学威海校区校草孙嘉良',
'泰山学院校草甄文忠',
'江汉大学校草曹文星',
'长沙理工大学城南学院校草朱泽贤',
'石家庄大学校草赵云龙',
'河南建筑职业技术学院校草杜健康',
'西南大学校草王壮',
'复旦校草胡瑞杰',
'临汾一中校草李江灜',
'清华校草詹凯',
'长沙理工大学城南学院校草毛创',
'东乡一中校草周凯波',
'鄱阳一中校草黄清',
'西南政法大学校草李帆',
'上杭职专校草曹永强',
'电子科技大学校草杨星',
'海南校草符是雄',
'曲阜远东职业技术学院校草王帅',
'西南交通大学校草路超',
'淮北师范大学校草刘家威',
'河北政法学院校草柏宇',
'鄱阳一中校草卢政辉',
'长沙理工大学城南学院校草朱泽贤'],
'url': ['/d/file/20150909/c76fa0a8c4d63f749fa3a6883bbdd4a8.jpg',
'/d/file/20161015/75541204516ea9e2118248fb374f706a.jpg',
'/d/file/7e191d68e7eb029b4c1da1a3808e5059.jpg',
'http://www.xiaohuar.com/d/file/20150512105953141.jpg',
'http://www.xiaohuar.com/d/file/20150813180033144.jpg',
'/d/file/63d5b955486e38d0fb664bcc9e8d96d6.jpg',
'/d/file/20160109/1b82558a72a175802de1c041d7c60813.jpg',
'/d/file/20170608/f07501a9d1fe9945e2fb96f0890a8d23.jpg',
'/d/file/20151018/e03c74b0e1fe13caab5e138c8b013a00.jpg',
'/d/file/9c9e3de9965854e46404af656d86a596.jpg',
'/d/file/180584665e90af45905e3ab2ab0bdb62.jpg',
'/d/file/f96fe78647d53691e5cb3d9f483d950e.jpg',
'http://www.xiaohuar.com/d/file/20140810073313127.jpg',
'/d/file/1b3b53eeae45059d3f46490d3eb45672.jpg',
'/d/file/8cb2bf048b0fdcbb8aabe6ca599eaa2b.jpg',
'/d/file/d02e95d32d1aed5fb46d5744ed5f9729.jpg',
'/d/file/65d5cfc0d5f54559c07073edf2421da7.jpg',
'/d/file/901b797f3ed0c5e1c7e861496ebe1c2d.jpg',
'/d/file/3047a6706453b987c26f54beba3e32ae.jpg',
'/d/file/20151003/9177cd57017a50783a94a4c700bae808.jpg',
'/d/file/b332da0c2a45f5ec08add990255b0071.jpg',
'http://www.xiaohuar.com/d/file/20150221160551154.jpg',
'http://www.xiaohuar.com/d/file/20150212230330155.jpg',
'/d/file/0540ae30cfbdf80a8121d5d6e14e8b62.jpg',
'/d/file/7e9a5cf7fd86a4afd8da51afe24cf82d.jpg',
'/d/file/ef1f9ff054246626f7a3edbcaab11810.jpg',
'/d/file/67af1100e5a0ba7e0de35f06e6d3c5fe.jpg',
'/d/file/4e0406f4073c4120b6e93cb08eafaf76.jpg',
'/d/file/20150915/f51d151afdd59c2f0b94ae5ebd9df2e5.jpg',
'/d/file/974244ea878d9d58cdeb5e581ba00c2d.png',
'/d/file/4bbf1aebc9e0ff735539e47185f07d86.jpg',
'/d/file/79f175dd8866f8bd756d0d663b56f63f.jpg',
'/d/file/20160128/edb010401a9314f51ba85dcef5628640.jpg',
'/d/file/38c8daefae96096777168b878ac12457.jpg',
'/d/file/f57c0b3d74190fcdf3496f29c495a9a8.jpg',
'/d/file/20161209/b0629634f424d3fc92ccf5e3311cf26d.jpg',
'/d/file/0d1c02b6d4eb2d49e06b85c920226e12.jpg',
'/d/file/20150917/1ea043c66124620e7bef39abfeaa7309.jpg',
'/d/file/ba9f9d31a9c1a74c0992fc53642351b4.jpg',
'/d/file/42d1ff3f5ee54c737a2a20691a51dd2e.jpg',
'/d/file/20151004/ff3c6b4a769142bcf07d36a38090e9ca.jpg',
'/d/file/54859a36a815d4cc9e4bfd0fa226a801.jpg',
'/d/file/20150829/61d547839957d38499af8a030f181bbc.jpg',
'/d/file/20170210/8e6c33aec8013aee65eb677f64ead2ff.jpg',
'/d/file/54da0d784895352c52c8de4c4724702b.jpg',
'/d/file/20151229/f7235e51a566a9b25343a3a034f47291.jpg',
'/d/file/ed3640812048582f544140c1dc29c701.jpg',
'/d/file/72edbf7ff56c981b82effd1dd2a181ff.jpg',
'/d/file/9aad0c431d4508d2924364532e5fb405.jpg',
'/d/file/6cc7ad88b47e1191cb112deed98a96cd.jpg',
'/d/file/0cb7df1d7ee292ddf6a7de7c9084e97c.jpg',
'/d/file/20170221/a6e051fd0752d6ad3beca2014a1f620e.jpg',
'/d/file/895782554cccd62f3819c0f912d8afba.jpg',
'/d/file/1a645a5896cc1a2989c9778c0dcd3d4e.jpg',
'/d/file/c077e29c5f9b89eb404505567df17c0c.jpg',
'/d/file/f825758f36e1c2c346f4f8f957df5430.jpg',
'/d/file/782cccaab95f8bdbcd6937c67cbaa3b0.jpg',
'/d/file/97f350f3e0fe65a8f7c768664e1eaca5.jpg',
'/d/file/883ea459b664518f60d9ea6fd9f49f0d.jpg',
'/d/file/281fd6f85bf33ace9b99b02031cb2cb9.jpg',
'/d/file/46cfceb6eac4bd6d5ab051098f413bdf.jpg',
'http://www.xiaohuar.com/d/file/20150221155450109.jpg',
'/d/file/c62ebcca53796107aaad2fe5577cde44.jpg',
'http://www.xiaohuar.com/d/file/20150505170150104.jpg',
'/d/file/2219bc2968708eb47ca47b618c96e003.jpg',
'/d/file/50c4400bda96f2305fc74de79aea913b.jpg',
'/d/file/cfdffb8e41273fbc287293bd2d0a3e59.jpg',
'/d/file/20150905/ffa892119baef28dadfcadc3b7abd50a.jpg',
'/d/file/20170331/5164f1dc347b2cd5d6b57088d2614c4d.jpg',
'/d/file/299823f15c021ede81f844b30b8d543a.jpg',
'/d/file/bf464364888cfecaf95b0a2a9e3407bf.png',
'/d/file/a9843ed50a56e020ed78e40a348bcb5c.jpg',
'/d/file/f9fe702b5e01210af5afd970e8499090.jpg',
'/d/file/d3e3d2526686463b4a1406e039388ca6.jpg',
'/d/file/1e29bc94ee7ebde0ae50bc2b5bd112c9.jpg',
'/d/file/db4d5e02fdea2e910ee35eb6b4979a99.jpg',
'/d/file/afaf7502ffca5a0bb052eddba8195509.jpg',
'/d/file/90d63585d7230a48ddd9516dbf0dd6ab.jpg',
'/d/file/689fa17c65c8d0f049521383d43a4ab2.jpg',
'/d/file/20151217/b5108ea8a7a4920b2d7bf6e27c0b1101.jpg',
'/d/file/e81bf99a232cdeee60dc65418a8c2602.png',
'/d/file/04c92ce23ee91e9cd857955e4a60b46d.jpg',
'/d/file/9a3315019ae4b15e84e18770f91e7f28.jpg',
'/d/file/2d64c4f98c23b4177ee1ae9725354ea4.jpg',
'/d/file/0ec2c52c6954837c2b588bb61e7e2e15.jpg',
'/d/file/20160220/b1e09b5d13c74f444806086a1a8aedbe.jpg',
'/d/file/6854f7987613ceac3a21049efa2f7047.jpg',
'/d/file/873316b54c2bc3bcc3573971fda73286.png',
'/d/file/117e999a87ef237115071d64b692b134.jpg',
'/d/file/24191b9f48d9f0ee58baf69e13a18c78.jpg',
'/d/file/2568b473a4a93b0be1366800d9194799.jpg',
'/d/file/0a83bccd885cba88a11eb93c5f789035.jpg',
'/d/file/a55cc2a57a2af7fad3477d799215ad05.jpg',
'/d/file/cd38314fd9c336afe56ef8d61cb8416d.jpg',
'/d/file/c0a77ff668ed2bb6292ec81a4c4105c7.jpg',
'/d/file/2bb8f36b6f540360d081ea17dad0d1c6.jpg',
'/d/file/20151213/e6f03eb1bb25208f230338ee76963d73.jpg',
'/d/file/c6bfce8e88fbeeb0ec94ed740ea02657.jpg',
'/d/file/9732937ef6b527bc9374bec637f209b4.jpg',
'/d/file/e0f286f6c4a83c5458c99e98d692a383.png']}