HTTPX: Dump requests library in a junkyard 🚀

The most common mistake in HTTP requests in Python is still using the requests library. The httpx library is a more advanced, versatile, and modern HTTP client for Python, supporting both synchronous and asynchronous workflows. It's easy to transition from the outdated requests library, which lacks critical features like native async support.

Install httpx:

pip install httpx

Async/Sync

httpx supports both synchronous and asynchronous clients, offering flexibility for various project environments.

• requests is sync-only, making it unsuitable for high-performance applications.
• aiohttp is async-only, which lacks simplicity for synchronous needs.

Synchronous Example

import httpx

# Synchronous GET request
response = httpx.get('https://api.example.com/data')
print(response.json())  # Parse JSON data

# Using a persistent Client for better performance
with httpx.Client(base_url='https://api.example.com') as client:
    response = client.get('/data')
    print(response.json())

Asynchronous Example

import httpx
import asyncio

async def fetch_data():
    # Asynchronous GET request
    async with httpx.AsyncClient(base_url='https://api.example.com') as client:
        response = await client.get('/data')
        print(response.json())

# Run the async function
asyncio.run(fetch_data())

Similar to requests

For those familiar with requests, transitioning to httpx is straightforward, as both libraries share similar APIs.

With requests:

import requests

response = requests.get('https://api.example.com/data')
data = response.json()

With httpx:

import httpx

response = httpx.get('https://api.example.com/data')
data = response.json()

Both libraries share similar method names (get, post, etc.), making migration easy.

Client Management

The concept of a Client in httpx is analogous to a Session in requests. However, httpx.Client is more powerful and efficient. You can read the article from the httpx documentation, to learn more about httpx.Client.

Features

• Performance: Clients reuse underlying connections, making multiple requests faster.
• Fine-grained control: Manage headers, cookies, timeouts, and other settings.
• Asynchronous support: Async clients (httpx.AsyncClient) are optimized for modern async workflows.

Perfomance Explanation

When you make a simple request, like the following:

import httpx

response = httpx.get('https://example-api.com', params={'page': 1})
print(response.json())

httpx follows this algorithm:

If you make dozens of requests, like this code:

import httpx

urls = [...]
params_list = [{...}, ...]

for url, params in zip(urls, params_list):
    response = httpx.get(url, params=params)
    print(response.json())

httpx will open the client for each request. It slows your application. The better solution is to use a single client instance. You can close it whenever you want.

import httpx

urls = [...]
params_list = [{...}, ...]

with httpx.Client() as client:
    for url, params in zip(urls, params_list):
        response = client.get(url, params=params)
        print(response.json())

Sensei Integration

The sensei Python framework uses httpx as its HTTP client for creating efficient, robust, and flexible API Clients (or API Wrappers). You can inject your own client into each request and close it whenever you want, to achieve the best performance:

from sensei import Manager, Router, Client

manager = Manager()
router = Router('httpx://example-api.com', manager=manager)

@router.get('/users/{id_}')
def get_user(id_: int) -> User:
    pass

with Client(base_url=router.base_url) as client:
    manager.set(client)

    for i in range(10):
        user = get_user(i)
        print(user)

    manager.pop()

Visit sensei repository to read more: https://github.com/CrocoFactory/sensei

Documentation

httpx provides rich, well-organized documentation site with examples and guides, making learning and troubleshooting easier and code docstrings explaining every function and parameter.

With these advantages and code examples, it's clear why httpx is the modern choice for HTTP requests in Python. Replace your requests code with httpx today and enjoy the benefits of a cutting-edge library! 🚀