How-to guide
How do you use an Asian residential proxy with Python Requests?
Short answerTo use a proxy with Python Requests, pass a proxies dictionary such as {"https": "http://USERNAME-country-jp:[email protected]:7777"} to each request. With Jade Proxies, the username also selects the city and a sticky session, and SOCKS5 works the same way with socks5h:// on port 7778.
What do you need before you start?
You need Python 3, the Requests library and proxy credentials. Jade Proxies supports username and password, or an IP allowlist (up to 10 IPs); sub-users with their own credentials and traffic cap; API keys you can revoke at any time. Every example sends traffic through the gateway gw.jadeproxies.com: port 7777 for HTTP and HTTPS, port 7778 for SOCKS5.
- 01
Install Requests with SOCKS support
Run
pip install "requests[socks]", which installs Requests and PySocks. - 02
Store your credentials
Put the proxy username and password from your Jade Proxies account in the environment variables
PROXY_USERandPROXY_PASS, not in the code. - 03
Build the HTTP proxy dictionary
Map both the
httpandhttpskeys tohttp://USER:[email protected]:7777; HTTPS sites are reached through a CONNECT tunnel, so TLS stays end to end. - 04
Switch to SOCKS5 when needed
Use
socks5h://USER:[email protected]:7778so DNS is resolved at the exit, in the target country. - 05
Target a country and a city
Extend the username with
-country-jp, then-city-osaka, following the formatUSERNAME-country-{cc}-city-{city}-session-{id}-ttl-{minutes}. - 06
Keep one IP with a sticky session
Add
-session-ID-ttl-MINUTESand reuse arequests.Session; the IP is kept up to 30 minutes (the IP can change earlier). - 07
Verify the exit IP
Request
https://ipinfo.io/jsonthrough the proxy and check thecountryfield before the real job. - 08
Set timeouts and retries
Pass
timeout=(10, 30)and mount anHTTPAdapterwith aRetrypolicy for 429 and 5xx answers. - 09
Run requests in parallel
Use a
ThreadPoolExecutor; one request per connection gives a new IP per request.
How do you set an HTTP proxy in Python Requests?
You set an HTTP proxy by passing a proxies dictionary whose http and https keys both point to the gateway's HTTP port.
import os
import requests
from urllib.parse import quote
USER = os.environ["PROXY_USER"]
PASS = quote(os.environ["PROXY_PASS"], safe="") # encodes @ : / in passwords
proxy = f"http://{USER}-country-jp:{PASS}@gw.jadeproxies.com:7777"
proxies = {"http": proxy, "https": proxy}
r = requests.get("https://example.com/", proxies=proxies, timeout=(10, 30))
print(r.status_code)
The proxy URL starts with http:// even under the https key: Requests opens a CONNECT tunnel through the gateway, and the TLS session runs between your script and the site. A 407 status means the gateway rejected the username or password.
How do you use a SOCKS5 proxy with Requests?
You use SOCKS5 by changing the scheme to socks5h and the port to 7778; the rest of the code stays the same.
proxy = f"socks5h://{USER}-country-kr:{PASS}@gw.jadeproxies.com:7778"
proxies = {"http": proxy, "https": proxy}
r = requests.get("https://example.com/", proxies=proxies, timeout=(10, 30))
With socks5h, the gateway resolves the hostname in Korea. With plain socks5, your own machine resolves it, which exposes the lookup to your local DNS and can return a server outside the target country.
How do you choose the country and the city?
You choose the exit by adding parameters to the username. Jade Proxies offers country and city targeting on every plan; ASN targeting at no extra cost; city availability varies.
| Goal | Username |
|---|---|
| Any IP in Japan | USER-country-jp |
| Osaka | USER-country-jp-city-osaka |
| Mumbai | USER-country-in-city-mumbai |
| Bangkok, same IP for 10 minutes | USER-country-th-city-bangkok-session-a1b2c3-ttl-10 |
Country codes are the two-letter ISO codes of the 13 countries on the Asian residential proxies page. A small helper keeps the format in one place:
def proxy_url(country, city=None, session=None, ttl=10, socks=False):
user = f"{USER}-country-{country}"
if city:
user += f"-city-{city}"
if session:
user += f"-session-{session}-ttl-{ttl}"
if socks:
return f"socks5h://{user}:{PASS}@gw.jadeproxies.com:7778"
return f"http://{user}:{PASS}@gw.jadeproxies.com:7777"
How do you keep the same IP across requests?
You keep the same IP by adding a session ID and a TTL to the username and sending the requests through one requests.Session.
import uuid
p = proxy_url("jp", "tokyo", session=uuid.uuid4().hex[:8], ttl=30)
s = requests.Session()
s.proxies = {"http": p, "https": p}
for page in range(1, 4):
s.get(f"https://example.jp/list?page={page}", timeout=(10, 30))
Without a session ID, each new connection gets an IP from the pool, usually a different one; to get a new IP on every request, send one request per connection (turn off keep-alive). A requests.Session keeps connections alive, so use plain requests.get() calls when you want a new IP on each request. The rotating vs sticky guide explains which mode fits which job.
How do you check that the exit IP is in the right country?
You check the exit by fetching an IP information endpoint through the proxy and comparing its country code with the one you requested.
info = requests.get("https://ipinfo.io/json", proxies=proxies, timeout=15).json()
assert info["country"] == "JP", info
print(info["ip"], info.get("city"), info.get("org"))
The org field names the network that owns the IP; for a residential exit it is a consumer ISP, not a hosting company. Run this check once per country before a long job.
How do you handle timeouts and retries?
You set a connect and a read timeout on every call and retry transient errors with backoff, because a residential exit is a home connection that can drop at any time.
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504], allowed_methods=["GET"])
s = requests.Session()
s.mount("http://", HTTPAdapter(max_retries=retry))
s.mount("https://", HTTPAdapter(max_retries=retry))
r = s.get("https://example.com/", proxies=proxies, timeout=(10, 30))
Retries cost traffic only when they reach the site: usage counts all data sent and received through the proxy (upload and download, headers included); 1 GB = 1,000,000,000 bytes; connection attempts that fail before reaching the target are not counted.
How do you send requests in parallel?
You send requests in parallel with a thread pool, because Requests is synchronous and each thread waits on its own connection.
from concurrent.futures import ThreadPoolExecutor
def fetch(url):
r = requests.get(url, proxies=proxies, timeout=(10, 30))
return url, r.status_code, len(r.content)
with ThreadPoolExecutor(max_workers=10) as pool:
for url, status, size in pool.map(fetch, urls):
print(status, size, url)
Each requests.get() call opens its own connection, so each request gets its own IP. Start with 10 workers, raise the number while the share of errors and 429 answers stays flat, and respect each site's rate limits and terms.
How much traffic does a Python Requests crawl use?
A crawl uses about the number of pages times the average transfer per page. The table assumes 100 KB per page: compressed HTML plus request and response headers, without images. That weight is an assumption for the calculation, not a measurement.
| Crawl | Traffic | Plan | Cost |
|---|---|---|---|
| 10,000 pages | 1 GB | Test | $1.50 |
| 100,000 pages | 10 GB | Growth | $6.60 |
| 500,000 pages | 50 GB | Pro | $26 |
| 1,000,000 pages | 100 GB | Business | $50 |
Our price grid shows that one million pages cost $50 in residential traffic on the 100 GB plan at that weight. Page weight varies by site, so measure your own average on a 1,000-page sample first; the Shopee and Lazada guide covers marketplace targets. All plans are on the pricing page, and traffic never expires.
Frequently asked questions
Why does Requests ignore my proxy?
Requests applies a proxy only to the URL schemes present in the proxies dictionary, so a dictionary with only an http key sends HTTPS requests directly. Set both keys, and check that a NO_PROXY environment variable does not match the target host.
What is the difference between socks5 and socks5h in Requests?
With socks5h, the proxy resolves the hostname at the exit; with socks5, your machine resolves it locally. Use socks5h for Asian targets so DNS answers match the exit country.
How do I get a new IP on every request?
Leave the session ID out of the username and send one request per connection, for example with plain requests.get() calls instead of a shared Session. Each new connection gets an IP from the pool, usually a different one.
What does a 407 error mean?
A 407 Proxy Authentication Required status means the gateway did not accept the credentials. Check the password encoding with urllib.parse.quote and the username syntax, for example USER-country-jp-city-osaka.
How much does a first test cost?
The 1 GB plan costs $1.50, about 10,000 pages at the 100 KB assumption above. It is paid from a crypto balance with a $25 minimum top-up, and unused traffic never expires.
Test 1 GB of Asian residential IPs for $1.50
Top up from $25 in BTC, ETH, USDT, USDC, SOL, LTC. No ID, no selfie. Unused balance never expires.