Python接口请求如何优化_批量请求与异常处理技巧【教学】

import requests
<p>session = requests.Session()</p><p><span>立即学习</span>“<a href="https://pan.quark.cn/s/00968c3c2c15" style="text-decoration: underline !important; color: blue; font-weight: bolder;" rel="nofollow" target="_blank">Python免费学习笔记（深入）</a>”；</p><h1>可选：调整连接池大小（避免TooManyRedirects或ConnectionPoolSizeError）</h1><p>adapter = requests.adapters.HTTPAdapter(pool_connections=20, pool_maxsize=20)
session.mount('http://', adapter)
session.mount('https://', adapter)</p><p>urls = ['<a href="https://www.php.cn/link/cf24f44a79866351337c1b317ffdc18d">https://www.php.cn/link/cf24f44a79866351337c1b317ffdc18d</a>', '<a href="https://www.php.cn/link/abc58d2523df2aea708a509fbd201437">https://www.php.cn/link/abc58d2523df2aea708a509fbd201437</a>']
for url in urls:
try:
resp = session.get(url, timeout=5)
resp.raise_for_status()
print(resp.json())
except requests.exceptions.RequestException as e:
print(f"请求失败 {url}：{e}")

from concurrent.futures import ThreadPoolExecutor, as_completed
<p>def fetch_one(session, url):
try:
resp = session.get(url, timeout=8)
resp.raise_for_status()
return {'url': url, 'status': 'success', 'data': resp.json()}
except Exception as e:
return {'url': url, 'status': 'error', 'error': str(e)}</p><h1>复用Session实例（注意：Session不是线程安全的，但用于GET基本无问题；如需绝对安全，可在每个worker里新建）</h1><p>with ThreadPoolExecutor(max_workers=10) as executor:
futures = [executor.submit(fetch_one, session, url) for url in urls]
for future in as_completed(futures):
result = future.result()
if result['status'] == 'success':
print("✅", result['url'])
else:
print("❌", result['url'], result['error'])

import time
import logging
from requests.exceptions import ConnectionError, Timeout, HTTPError
<p>def safe_get(session, url, max_retries=2, backoff_factor=1):
for i in range(max_retries + 1):
try:
resp = session.get(url, timeout=10)
resp.raise_for_status()
return resp
except ConnectionError:
if i == max_retries:
raise
logging.warning(f"连接失败 {url}，{backoff_factor * (2 <strong> i)}s后重试")
time.sleep(backoff_factor * (2 *<em> i))
except Timeout:
if i == max_retries:
raise
logging.warning(f"超时 {url}，重试中...")
time.sleep(backoff_factor </em> (2 </strong> i))
except HTTPError as e:
if resp.status_code >= 500 and i < max_retries:
logging.warning(f"服务端错误 {url}（{resp.status_code}），重试...")
time.sleep(backoff_factor * (2 ** i))
continue
raise  # 4xx直接抛出