CalvinBackup/openproxy
1
0
Fork 0
mirror of https://github.com/praveentcom/openproxy.git synced 2026-07-17 17:37:22 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat: update configuration for OpenAI and Anthropic endpoints

Browse Source 
- Created a new .env.example file with default environment variables for PORT, OPENAI_UPSTREAM_URL, ANTHROPIC_UPSTREAM_URL, and DATABASE_URL.
- Updated .npmignore to exclude all .env files except .env.example.
- Revised CONTRIBUTING.md to simplify the contribution process and provide clearer setup instructions.
- Enhanced cost.ts with detailed type definitions and improved cost calculation logic.
- Updated proxy.ts to include new environment variables and improved logging functionality.
- Modified README.md to reflect new configuration instructions and usage examples.
- Removed unnecessary dashboard files and streamlined the project structure.
This commit is contained in:
Praveen Thirumurugan
2025-12-23 12:37:40 +05:30
parent e5231cac8c
commit 18d4c93216
17 changed files with 1968 additions and 2653 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+77 -126
View File
@@ -1,42 +1,66 @@
# OpenProxy
# ✨ OpenProxy: Multi-provider LLM proxy with cost tracking
A lightweight, production-ready OpenAI-compatible proxy server that seamlessly forwards LLM API requests to any endpoint with comprehensive logging, cost tracking, and PostgreSQL integration. Perfect for monitoring API usage, calculating costs, and maintaining audit trails for your AI applications.
OpenProxy is a lightweight, production-ready proxy server that seamlessly forwards API requests to OpenAI and Anthropic compatible endpoints with comprehensive logging, cost tracking, and PostgreSQL integration.
<img width="2400" height="1260" alt="cover-image" src="https://github.com/user-attachments/assets/60f6e577-7d2e-45d0-accb-3af62894840f" />
## ⚙️ Configuration
## How to configure?
| Environment Variable | Description | Default Value |
|----------------------|-------------|-----------------|
| `PORT` | Server port | `3007` |
| `UPSTREAM_URL` | Your LLM endpoint URL | **Required** |
| `DATABASE_URL` | PostgreSQL connection string for logging | **Required** |
| `DATABASE_TABLE` | Name of the table to store the logs | `"llm_proxy"` |
### Setting up
## 💰 Cost Calculation
I'd recommend forking this repository or cloning it directly. Once done, you should be able to get OpenProxy running with minimal configuration.
The cost is calculated based on the model and token usage with configurable pricing per model.
You'll need to add the cost configuration (in cost per million tokens) for your models in the `cost.ts` file. The default cost configuration in the project (with sample values from `z.ai` models) is:
```typescript
export const MODEL_COSTS: Record<string, CostConfig> = {
"glm-4.5-air": { input: 0.2, cached: 0.03, output: 1.1 },
"glm-4.6": { input: 0.6, cached: 0.11, output: 2.2 },
"default": { input: 0, cached: 0, output: 0 },
};
```bash
pnpm install
```
You can add more models to the `MODEL_COSTS` object to support your specific LLM providers.
Set your environment variables:
## 📊 PostgreSQL Table Schema
```bash
export PORT=3007
export OPENAI_UPSTREAM_URL="https://api.example.com/v1"
export ANTHROPIC_UPSTREAM_URL="https://api.example.com/api/anthropic/v1"
export DATABASE_URL="postgresql://user:password@localhost:5432/database_name"
```
Start the server:
```bash
# Development mode with auto-reload
pnpm dev
# Production build
pnpm build && pnpm start
```
### Configuration
| Environment Variable | Description | Default |
|----------------------|-------------|---------|
| `PORT` | Server port | `3007` |
| `OPENAI_UPSTREAM_URL` | OpenAI-compatible endpoint URL | - |
| `ANTHROPIC_UPSTREAM_URL` | Anthropic-compatible endpoint URL | - |
| `DATABASE_URL` | PostgreSQL connection string | **Required** |
OpenProxy uses path prefixes for clean provider detection:
| Proxy Path | Routes To | Auth Header |
|------------|-----------|-------------|
| `/openai/*` | `OPENAI_UPSTREAM_URL/*` | `Authorization: Bearer <key>` |
| `/anthropic/*` | `ANTHROPIC_UPSTREAM_URL/*` | `x-api-key: <key>` or `Authorization: Bearer <key>` |
### PostgreSQL Logging
Every request is logged with comprehensive details to the PostgreSQL database. The table schema is as follows:
```sql
CREATE TABLE IF NOT EXISTS <DATABASE_TABLE> (
CREATE TABLE IF NOT EXISTS llm_proxy (
timestamp TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
request_method VARCHAR(10) NOT NULL,
request_path VARCHAR(255) NOT NULL,
model VARCHAR(20) NOT NULL,
provider TEXT,
model VARCHAR(50) NOT NULL,
completion_tokens INTEGER,
prompt_tokens INTEGER,
total_tokens INTEGER,
@@ -56,123 +80,50 @@ CREATE TABLE IF NOT EXISTS <DATABASE_TABLE> (
max_tokens INTEGER,
request_id UUID
);
CREATE INDEX IF NOT EXISTS idx_<DATABASE_TABLE>_timestamp ON <DATABASE_TABLE> (timestamp);
CREATE INDEX IF NOT EXISTS idx_<DATABASE_TABLE>_request_id ON <DATABASE_TABLE> (request_id);
```
## 🚀 Quick Start
### Cost Calculation
### Installation
OpenProxy automatically calculates costs based on model and token usage using the Helicone API. You can customize the costs for your own models in `cost.ts`.
## How to use?
### Using with Claude Code
For example, to use Z.AI or other Anthropic-compatible providers with Claude Code:
```bash
npm install
export ANTHROPIC_UPSTREAM_URL="https://api.z.ai/api/anthropic"
export DATABASE_URL="postgresql://user:password@localhost:5432/database_name"
pnpm dev
# Configure Claude Code to use:
# API Base URL: http://localhost:3007/anthropic
```
### Configuration
### Using with OpenAI-compatible clients
Set your environment variables:
For example, to use Z.AI or other OpenAI-compatible providers with OpenAI-compatible clients:
```bash
export PORT=3007
export UPSTREAM_URL="https://api.example.com/v1"
export DATABASE_URL="postgresql://user:password@localhost:5432/llm_logs"
export DATABASE_TABLE="llm_proxy"
export OPENAI_UPSTREAM_URL="https://api.z.ai/api/coding/paas/v4"
export DATABASE_URL="postgresql://user:password@localhost:5432/database_name"
pnpm dev
# Configure your client to use:
# API Base URL: http://localhost:3007/openai
```
### Running
## Metrics Dashboard
OpenProxy includes a lightweight Next.js dashboard for real-time metrics visualization. The dashboard is accessible at `http://localhost:3008`. To run the dashboard, run the following command:
```bash
# Development mode with auto-reload
npm run dev
# Production build
npm run build
npm start
pnpm --filter dashboard dev
```
## 💻 Usage
## Final notes
The proxy works with any OpenAI-compatible endpoint. Just point your client to the proxy:
Get started today and for assistance, you can reach me on [GitHub](https://github.com/praveentcom/openproxy) or [X](https://x.com/praveentcom). PRs are always welcome if you'd like to add features or fix bugs!
```bash
curl -X POST http://localhost:3007/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-api-key" \
-d '{
"model": "your-model",
"messages": [{"role": "user", "content": "Hello!"}]
}'
```
### Example Response with Cost Tracking
All responses are logged to PostgreSQL with detailed usage and cost information:
```json
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1677652288,
"model": "glm-4.5-air",
"usage": {
"prompt_tokens": 20,
"completion_tokens": 30,
"total_tokens": 50,
"prompt_tokens_details": {
"cached_tokens": 5
}
},
"choices": [...]
}
```
The corresponding database entry will include:
- Token usage breakdown
- Calculated cost based on your model pricing
- Response time metrics
- Complete request/response bodies for audit purposes
## 🛡️ Security
- Bearer token authentication required
- CORS headers configured for cross-origin requests
- No sensitive data stored in logs (authentication headers are not logged)
- Input validation and error handling
## 📈 Monitoring
Monitor your API usage through the PostgreSQL logs:
- Track costs across different models
- Analyze response times and performance
- Identify usage patterns and optimize costs
- Maintain compliance with audit requirements
### Metrics Dashboard
OpenProxy includes a lightweight Next.js dashboard for real-time metrics visualization:
```bash
cd dashboard
npm install
cp .env.example .env
# Configure DATABASE_URL in .env
npm run dev
```
The dashboard (available at `http://localhost:3008`) provides:
- **Real-time Overview**: Total requests, tokens, costs, and response times
- **Model Breakdown**: Usage statistics grouped by LLM model
- **Hourly Trends**: Visual charts showing request patterns over time
- **Recent Requests**: Detailed table of recent API calls
- **Auto-refresh**: Automatic updates every 30 seconds
See [dashboard/README.md](./dashboard/README.md) for detailed setup instructions.
## 🤝 Contributing
Feel free to submit issues and enhancement requests!
## 📄 License
This project is open source and available under the MIT License.
This project is open source and available under the [MIT License](./LICENSE).