Crate object_storage_proxy 

object-storage-proxy

A fast, in-process reverse proxy for AWS S3, IBM Cloud Object Storage (COS) and other S3-compatible object storage services, with a Python interface for custom authentication and credential management.

The proxy is built on top of Pingora and exposed to Python via PyO3. It handles:

• AWS Signature Version 4 re-signing — incoming requests are validated and then re-signed with backend credentials before being forwarded.
• Presigned URL enforcement — optional per-URL usage limits prevent replay abuse.
• IBM IAM bearer-token exchange — API keys are automatically exchanged for short-lived IAM tokens and cached.
• Pluggable Python callbacks — supply an async validator and/or a credential fetcher callable from Python to integrate with any auth backend.

Quick start (Python)

from object_storage_proxy import ProxyServerConfig, start_server

config = ProxyServerConfig(
    cos_map={
        "my-bucket": {
            "host": "s3.eu-west-3.amazonaws.com",
            "port": 443,
            "access_key": "AKIAIOSFODNN7EXAMPLE",
            "secret_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
            "region": "eu-west-3",
        }
    },
    http_port=6190,
)
start_server(config)

Modules

credentials

Credential types, signing, and secret management.

parsers

Nom-based parsers for paths, query strings, credentials, and the COS map.

utils

Utility helpers shared across the proxy.

Structs

MyCtx

Per-request context threaded through the Pingora middleware chain.

MyProxy

The core Pingora proxy handler.

ProxyServerConfig

Configuration object for :pyfunc:object_storage_proxy.start_server.

UrlTracker

Thread-safe hit counter for presigned URLs.

Constants

DEFAULT_SERVER_NAME 🔒

Statics

REQ_COUNTER 🔒

REQ_COUNTER_ENABLED 🔒

Functions

__pyfunction_disable_request_counting 🔒 ^⚠

__pyfunction_enable_request_counting 🔒 ^⚠

__pyfunction_get_request_count 🔒 ^⚠

__pyfunction_start_server 🔒 ^⚠

disable_request_counting 🔒

Disable the global request counter.

enable_request_counting 🔒

Enable the global request counter (disabled by default).

get_request_count 🔒

Return the total number of proxied requests since counting was enabled.

init_tracing

Initialise the global [tracing] subscriber.

object_storage_proxy 🔒

run_server

Build and run the Pingora proxy server.

start_server

Start an HTTP + HTTPS reverse‑proxy for IBM COS.