溫馨提示×
您好，登錄后才能下訂單哦！
點擊 登錄注冊 即表示同意《億速云用戶服務條款》
您好，登錄后才能下訂單哦！
這篇“python怎么實現多線程并得到返回值”文章的知識點大部分人都不太理解，所以小編給大家總結了以下內容，內容詳細，步驟清晰，具有一定的借鑒價值，希望大家閱讀完這篇文章能有所收獲，下面我們一起來看看這篇“python怎么實現多線程并得到返回值”文章吧。
# -*- coding:utf-8 -*-
"""
作者：wyt
日期：2022年04月21日
"""
import threading
import requests
import time
urls = [
f'https://www.cnblogs.com/#p{page}' # 待爬地址
for page in range(1, 10) # 爬取1-10頁
]
def craw(url):
r = requests.get(url)
num = len(r.text) # 爬取博客園當頁的文字數
return num # 返回當頁文字數
def sigle(): # 單線程
res = []
for i in urls:
res.append(craw(i))
return res
class MyThread(threading.Thread): # 重寫threading.Thread類，加入獲取返回值的函數
def __init__(self, url):
threading.Thread.__init__(self)
self.url = url # 初始化傳入的url
def run(self): # 新加入的函數，該函數目的：
self.result = craw(self.url) # ①。調craw(arg)函數，并將初試化的url以參數傳遞——實現爬蟲功能
# ②。并獲取craw(arg)函數的返回值存入本類的定義的值result中
def get_result(self): #新加入函數，該函數目的：返回run()函數得到的result
return self.result
def multi_thread():
print("start")
threads = [] # 定義一個線程組
for url in urls:
threads.append( # 線程組中加入賦值后的MyThread類
MyThread(url) # 將每一個url傳到重寫的MyThread類中
)
for thread in threads: # 每個線程組start
thread.start()
for thread in threads: # 每個線程組join
thread.join()
list = []
for thread in threads:
list.append(thread.get_result()) # 每個線程返回結果(result)加入列表中
print("end")
return list # 返回多線程返回的結果組成的列表
if __name__ == '__main__':
start_time = time.time()
result_multi = multi_thread()
print(result_multi) # 輸出返回值-列表
# result_sig = sigle()
# print(result_sig)
end_time = time.time()
print('用時：', end_time - start_time)單線程：
多線程：
加速效果明顯。
import threading
import requests
import time
urls = [
f'https://www.cnblogs.com/#p{page}' # 待爬地址
for page in range(1, 10) # 爬取1-10頁
]
def craw(url):
r = requests.get(url)
num = len(r.text) # 爬取博客園當頁的文字數
print(num)
def sigle(): # 單線程
res = []
for i in urls:
res.append(craw(i))
return res
def multi_thread():
print("start")
threads = [] # 定義一個線程組
for url in urls:
threads.append(
threading.Thread(target=craw,args=(url,)) # 注意args=(url,)，元組
)
for thread in threads: # 每個線程組start
thread.start()
for thread in threads: # 每個線程組join
thread.join()
print("end")
if __name__ == '__main__':
start_time = time.time()
result_multi = multi_thread()
# result_sig = sigle()
# print(result_sig)
end_time = time.time()
print('用時：', end_time - start_time)返回：
start
69915
69915
69915
69915
69915
69915
69915
69915
69915
end
用時： 0.316709041595459
import time from threading import Thread def foo(number): time.sleep(1) return number class MyThread(Thread): def __init__(self, number): Thread.__init__(self) self.number = number def run(self): self.result = foo(self.number) def get_result(self): return self.result if __name__ == '__main__': thd1 = MyThread(3) thd2 = MyThread(5) thd1.start() thd2.start() thd1.join() thd2.join() print(thd1.get_result()) print(thd2.get_result())
返回：
3
5
多線程入口
threading.Thread(target=craw,args=(url,)) # 注意args=(url,)，元組
多線程傳參
需要重寫一下threading.Thread類，加一個接收返回值的函數。 三、代碼實戰
使用這種帶返回值的多線程技術重寫了一下之前發布過的一個爬取子域名的代碼，原始代碼在這里：https://blog.csdn.net/qq_45859826/article/details/124030119
import threading
import requests
from bs4 import BeautifulSoup
from static.plugs.headers import get_ua
#https://cn.bing.com/search?q=site%3Abaidu.com&go=Search&qs=ds&first=20&FORM=PERE
def search_1(url):
Subdomain = []
html = requests.get(url, stream=True, headers=get_ua())
soup = BeautifulSoup(html.content, 'html.parser')
job_bt = soup.findAll('h3')
for i in job_bt:
link = i.a.get('href')
# print(link)
if link not in Subdomain:
Subdomain.append(link)
return Subdomain
class MyThread(threading.Thread):
def __init__(self, url):
threading.Thread.__init__(self)
self.url = url
def run(self):
self.result = search_1(self.url)
def get_result(self):
return self.result
def Bing_multi_thread(site):
print("start")
threads = []
for i in range(1, 30):
url = "https://cn.bing.com/search?q=site%3A" + site + "&go=Search&qs=ds&first=" + str(
(int(i) - 1) * 10) + "&FORM=PERE"
threads.append(
MyThread(url)
)
for thread in threads:
thread.start()
for thread in threads:
thread.join()
res_list = []
for thread in threads:
res_list.extend(thread.get_result())
res_list = list(set(res_list)) #列表去重
number = 1
for i in res_list:
number += 1
number_list = list(range(1, number + 1))
dict_res = dict(zip(number_list, res_list))
print("end")
return dict_res
if __name__ == '__main__':
print(Bing_multi_thread("qq.com"))返回：
{
1:'https://transmart.qq.com/index',
2:'https://wpa.qq.com/msgrd?v=3&uin=448388692&site=qq&menu=yes',
3:'https://en.exmail.qq.com/',
4:'https://jiazhang.qq.com/wap/com/v1/dist/unbind_login_qq.shtml?source=h6_wx',
5:'http://imgcache.qq.com/',
6:'https://new.qq.com/rain/a/20220109A040B600',
7:'http://cp.music.qq.com/index.html',
8:'http://s.syzs.qq.com/',
9:'https://new.qq.com/rain/a/20220321A0CF1X00',
10:'https://join.qq.com/about.html',
11:'https://live.qq.com/10016675',
12:'http://uni.mp.qq.com/',
13:'https://new.qq.com/omn/TWF20220/TWF2022042400147500.html',
14:'https://wj.qq.com/?from=exur#!',
15:'https://wj.qq.com/answer_group.html',
16:'https://view.inews.qq.com/a/20220330A00HTS00',
17:'https://browser.qq.com/mac/en/index.html',
18:'https://windows.weixin.qq.com/?lang=en_US',
19:'https://cc.v.qq.com/upload',
20:'https://xiaowei.weixin.qq.com/skill',
21:'http://wpa.qq.com/msgrd?v=3&uin=286771835&site=qq&menu=yes',
22:'http://huifu.qq.com/',
23:'https://uni.weixiao.qq.com/',
24:'http://join.qq.com/',
25:'https://cqtx.qq.com/',
26:'http://id.qq.com/',
27:'http://m.qq.com/',
28:'https://jq.qq.com/?_wv=1027&k=pevCjRtJ',
29:'https://v.qq.com/x/page/z0678c3ys6i.html',
30:'https://live.qq.com/10018921',
31:'https://m.campus.qq.com/manage/manage.html',
32:'https://101.qq.com/',
33:'https://new.qq.com/rain/a/20211012A0A3L000',
34:'https://live.qq.com/10021593',
35:'https://pc.weixin.qq.com/?t=win_weixin&lang=en',
36:'https://sports.qq.com/lottery/09fucai/cqssc.htm'
}
非常非常非常能感受到速度快了超級多，用這種方式爆破子域名也比較爽。沒有多線程，我的項目里可能缺少了好幾個功能：因為之前寫過的一些程序都因執行時間過長被我砍掉。這個功能還是很實用的。
以上就是關于“python怎么實現多線程并得到返回值”這篇文章的內容，相信大家都有了一定的了解，希望小編分享的內容對大家有幫助，若想了解更多相關的知識內容，請關注億速云行業資訊頻道。
免責聲明：本站發布的內容（圖片、視頻和文字）以原創、轉載和分享為主，文章觀點不代表本網站立場，如果涉及侵權請聯系站長郵箱：is@yisu.com進行舉報，并提供相關證據，一經查實，將立刻刪除涉嫌侵權內容。
