当通过python爬虫爬取外网资源,python需要通过socks5等代理连接网络,这里记录一下两种常用方法。
修改内置py库
当使用urllib或者socket等内置库连接网络时,分别修改urllib.py或者socket.py文件。
不过修改以后,再使用其他python程序时也会受到影响,需要记得再改回来。
或者使用pyenv或者类似的工具,使该程序使用独立的虚拟python环境,防止影响其他程序。
修改urllib.pyimport socks import socket socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 1080) socket.socket = socks.socksocket
修改socket.py
socket = SocketType = _socketobject # 在socket.py找到这一行 # 在下面添加下面的代码 import socks socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 1080) socket = socks.socksocket
使用PySocks
PySocks是一个基于Python的SOCKS代理客户端,它是SocksiPy的一个分支,修改了一些bug和增加了一些额外功能。
安装
git clone https://github.com/Anorov/PySocks cd PySocks python setup.py install
或者直接pip install PySocks
使用
import socket import socks socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 1080) socket.socket = socks.socksocket response = request.get("www.google.com")
当众人都哭时,
应该允许有的人不哭。
当哭成为一种表演时,
更应该允许有的人不哭。
——莫言
评论
What is Alpha Tonic? Alpha Tonic stands as a natural health supplement designed to comprehensively address men’s overall well-being.
https://youtu.be/OmJBZa9atjc
Thank you for sharing superb informations. Your site is very cool. I’m impressed by the details that you’ve on this website. It reveals how nicely you perceive this subject. Bookmarked this web page, will come back for more articles. You, my pal, ROCK! I found just the information I already searched all over the place and just couldn’t come across. What an ideal web site.
https://youtu.be/7IDhVlV6-38
I’d forever want to be update on new articles on this site, saved to favorites! .
https://youtu.be/2hfKFiLKaFg
I like what you guys are up also. Such intelligent work and reporting! Carry on the excellent works guys I have incorporated you guys to my blogroll. I think it’ll improve the value of my website 🙂
https://youtu.be/BanYFY6l55c
49353 804118I like the way you conduct your posts. Hmm 821836
It is really a nice and useful piece of info. I am glad that you simply shared this useful information with us. Please keep us informed like this. Thanks for sharing.
https://youtu.be/mykohPj9o34
Appreciate it for this marvelous post, I am glad I discovered this web site on yahoo.
https://youtu.be/C4K3FEdRfYA
This is really interesting, You’re a very skilled blogger. I’ve joined your rss feed and look forward to seeking more of your excellent post. Also, I’ve shared your web site in my social networks!
https://youtu.be/sE6KesB4jDs