Configure proxies, retries, and sessions in ytscrape¶
Inject a custom requests.Session into InnerTubeClient to configure proxies, automatic retries, timeouts, and testable fake HTTP backends.
YouTube is a thin facade over InnerTubeClient, which owns the requests.Session that performs every HTTP call. Injecting your own session is the single hook for proxies, retries, custom headers, caching, or unit-test mocks — none of the rest of the library needs to change.
Injecting a custom session¶
Build your requests.Session, configure it, then wrap it in an InnerTubeClient and pass that to YouTube:
import requests
from ytscrape import YouTube, InnerTubeClient
session = requests.Session()
# ... configure session here ...
client = InnerTubeClient(session=session, language="en", region="US")
with YouTube(client=client) as yt:
print(next(iter(yt.search("python"))).title)
Any option you set on the session — headers, auth, cookies, adapters — is automatically used for every request ytscrape makes.
Proxies¶
The quickest way to route every request through a proxy is the built-in
proxies argument. Pass a requests-style mapping straight to YouTube
(or InnerTubeClient) — no custom session required:
from ytscrape import YouTube
proxy = {"https": "http://user:pass@proxy:8080"}
# or route both schemes:
# proxy = {
# "http": "http://user:pass@proxy:8080",
# "https": "http://user:pass@proxy:8080",
# }
with YouTube(proxies=proxy) as yt:
details = yt.video("https://www.youtube.com/watch?v=dQw4w9WgXcQ")
print(details.title)
Rotating proxies (after a bot block)¶
Because YouTube may start serving captchas once it flags an IP, you can swap
the proxy at runtime by assigning to the proxies property. This is handy when
you catch a BotDetected / RateLimited error and want to retry from a fresh
IP without rebuilding YouTube:
from ytscrape import YouTube, BotDetected, RateLimited
proxies = [
{"https": "http://user:pass@proxy-a:8080"},
{"https": "http://user:pass@proxy-b:8080"},
]
with YouTube(proxies=proxies[0]) as yt:
for attempt, proxy in enumerate(proxies):
yt.proxies = proxy # rotate to the next IP
try:
details = yt.video("dQw4w9WgXcQ")
break
except (BotDetected, RateLimited):
if attempt == len(proxies) - 1:
raise
Via a custom session¶
For finer control (SOCKS proxies, per-scheme adapters, cookies) you can still
inject your own session and set session.proxies on it:
import requests
from ytscrape import YouTube, InnerTubeClient
session = requests.Session()
session.proxies = {"https": "http://user:pass@proxy:8080"}
client = InnerTubeClient(session=session, language="en", region="US")
with YouTube(client=client) as yt:
print(next(iter(yt.search("python"))).title)
Retries with HTTPAdapter¶
Mount an HTTPAdapter with a Retry policy to automatically retry transient failures:
import requests
from requests.adapters import HTTPAdapter, Retry
from ytscrape import YouTube, InnerTubeClient
session = requests.Session()
session.proxies = {"https": "http://user:pass@proxy:8080"}
session.mount(
"https://",
HTTPAdapter(max_retries=Retry(total=5, backoff_factor=1)),
)
client = InnerTubeClient(session=session, timeout=15.0, language="en", region="US")
with YouTube(client=client) as yt:
print(next(iter(yt.search("python"))).title)
backoff_factor=1 means the client waits 1 s, 2 s, 4 s, … between retries. See the urllib3 Retry documentation for the full set of options.
Custom timeout¶
Pass timeout (in seconds) to InnerTubeClient to override the default of 30 seconds. This timeout applies to every individual request:
client = InnerTubeClient(timeout=10.0, language="en", region="US")
with YouTube(client=client) as yt:
details = yt.video("dQw4w9WgXcQ")
You can also set timeout directly on the default client by passing it to YouTube:
Using InnerTubeClient directly¶
InnerTubeClient exposes the raw InnerTube endpoint wrappers (search, player, browse, next) when you need lower-level access. All of them return the parsed JSON dictionary from YouTube's API:
from ytscrape import InnerTubeClient
with InnerTubeClient(language="en", region="US") as client:
data = client.player("dQw4w9WgXcQ")
print(data["videoDetails"]["title"])
Pass a fully configured InnerTubeClient to YouTube when you want to share the same session and context across all high-level calls:
Testing with a fake session¶
Because InnerTubeClient accepts any requests.Session, you can inject a mock in tests so that no real network call ever happens:
from unittest.mock import MagicMock
import requests
from ytscrape import YouTube, InnerTubeClient
fake_session = MagicMock(spec=requests.Session)
# Configure fake_session.get / .post to return canned responses...
client = InnerTubeClient(session=fake_session)
yt = YouTube(client=client)
# All calls on `yt` now go through your mock.
Tip
Reuse a single YouTube instance across multiple operations. On the first call, the client fetches the YouTube home page to extract the InnerTube context (API key, client version, visitor data). Reusing the same instance keeps that context warm and avoids an extra round-trip on every subsequent call.
Warning
requests.Session is not thread-safe. If you run ytscrape from multiple threads, create a separate YouTube (and therefore a separate session) per thread rather than sharing one instance.
Async client (httpx)¶
For asyncio apps, use AsyncYouTube / AsyncInnerTubeClient instead of injecting a requests.Session. Concurrency and retries are first-class constructor arguments:
import asyncio
import httpx
from ytscrape import AsyncYouTube, AsyncInnerTubeClient
async def main() -> None:
session = httpx.AsyncClient(proxy="http://user:pass@proxy:8080")
client = AsyncInnerTubeClient(
session=session,
max_concurrency=8,
max_retries=3,
backoff_factor=0.5,
timeout=15.0,
)
async with AsyncYouTube(client=client) as yt:
details = await yt.video("dQw4w9WgXcQ")
print(details.title)
asyncio.run(main())
Or configure the default client without building httpx yourself:
See the full Async API guide for pagination, multi-video asyncio.gather patterns, and install instructions (pip install "ytscrape[async]").