Getting Started
Prerequisites
Docker 24+
For containerised install
Python 3.10+
For SDK / CLI
Node 18+
For JS/TS SDK
Install
docker pull g8kepr/api:latest
docker run -d \
-e API_KEY=your_api_key \
-e DATABASE_URL=postgresql://user:pass@db:5432/g8kepr \
-p 8000:8000 \
--name g8kepr \
g8kepr/api:latestAPI key security
First Request
curl -X POST https://api.g8kepr.com/v1/gateway/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [{ "role": "user", "content": "Hello from G8KEPR!" }]
}'{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"model": "gpt-4o-mini",
"choices": [{ "message": { "role": "assistant", "content": "Hello! How can I help?" } }],
"g8kepr": {
"threat_score": 0.01,
"latency_ms": 312,
"gateway_id": "gw_prod_01"
}
}SDK Setup
from g8kepr import G8KPRClient
client = G8KPRClient(api_key="YOUR_API_KEY")
response = client.gateway.complete(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Explain zero-trust security"}],
)
print(response.choices[0].message.content)Architecture Overview
System Overview
G8KEPR Architecture Overview
OWASP Top 10
AI Tool Control
Multi-LLM Routing
Metrics & Logs
Request Flow
Request Flow Through G8KEPR
Processing Layers
01
Ingress
TLS termination, IP allowlist, SNI routing
02
Gateway Proxy
Rate limiting, auth validation, idempotency
03
Threat Engine
1,500+ patterns, session correlation, ML scoring
04
Backend Forward
mTLS, retry, circuit-breaker, audit log
Deployment Options
Single Node
Development & small teams. Docker Compose, 1 server.
Patroni HA
Production HA with automatic failover. PostgreSQL primary + 2 replicas.
Kubernetes
Enterprise scale. Helm chart, HPA, PodDisruptionBudget, zone-aware scheduling.
Docker Compose
Docker Deployment Architecture
Kubernetes / Helm
Kubernetes Deployment
API Security
Live Request Monitor
HTTP Integration
Direct HTTP Integration
api.g8kepr.com/v1OpenAI-compatiblepip install g8kepr-cliOWASP Top 10 Coverage
Broken Access Control
Cryptographic Failures
Injection
Insecure Design
Security Misconfiguration
Vulnerable Components
Auth & Identity Failures
Data Integrity Failures
Security Logging Failures
SSRF
Rate Limiting Tiers
| Plan | Req / min | Req / day | Burst | Concurrent |
|---|---|---|---|---|
| Free | 60 | 10,000 | 10 | 5 |
| Pro | 1,000 | 500,000 | 200 | 50 |
| Enterprise | Unlimited | Unlimited | 5,000 | Unlimited |
Geo-Blocking Config
{
"rule_name": "block-high-risk-countries",
"action": "block",
"countries": ["CN", "RU", "KP", "IR"],
"response_code": 403,
"log": true
}Rate limit headers
X-RateLimit-Limit, X-RateLimit-Remaining, and Retry-After on 429. Implement exponential backoff — start at 1s, double each retry, cap at 32s.MCP Security
MCP Security Flow
Session Tracking
Include X-Session-ID in every request to enable multi-turn threat correlation. G8KEPR tracks escalation patterns across requests within a session.
# All requests in a session share the same X-Session-ID
curl -X POST https://api.g8kepr.com/v1/gateway/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "X-Session-ID: sess_abc123xyz" \
-H "Content-Type: application/json" \
-d '{ "model": "gpt-4o", "messages": [...] }'Tool Authorization — RBAC
| Tool Category | viewer | operator | admin | super_admin |
|---|---|---|---|---|
| Read files | ✓ | ✓ | ✓ | ✓ |
| Write files | ✗ | ✓ | ✓ | ✓ |
| Execute code | ✗ | ✗ | ✓ | ✓ |
| Network access | ✗ | ✗ | ✗ | ✓ |
| System calls | ✗ | ✗ | ✗ | ✓ |
Explainability Response
{
"allowed": false,
"threat_score": 0.87,
"decision": "block",
"explanation": "Prompt injection pattern detected: role confusion + instruction override",
"patterns_matched": [
{ "id": "PI-001", "name": "Role Confusion", "confidence": 0.92 },
{ "id": "PI-007", "name": "Instruction Override", "confidence": 0.81 }
],
"session_escalation_factor": 2.0,
"session_request_count": 4
}Audit Log Query
curl https://api.g8kepr.com/v1/mcp/audit?session_id=sess_abc123xyz \
-H "Authorization: Bearer YOUR_API_KEY"Session expiry
/v1/mcp/sessions/{id}/heartbeat — this resets the 30-minute clock without consuming quota.AI Gateway
AI Gateway - Intelligent Routing
Cost Savings
Uptime
Latency Added
Routing Strategies
Distributes requests evenly across all healthy providers. Good for load balancing when all providers have similar SLAs.
{
"strategy": "round_robin",
"providers": ["openai", "anthropic", "cohere"],
"health_check_interval_s": 30
}Model Marketplace
OpenAI
gpt-4o
gpt-4o-mini
o1
Anthropic
claude-opus-4-6
claude-sonnet-4-6
claude-haiku-4-5
Cohere
command-r-plus
command-r
embed-v3
Streaming
import httpx
with httpx.stream("POST", "https://api.g8kepr.com/v1/gateway/completions",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={"model": "gpt-4o", "messages": [...], "stream": True},
) as resp:
for line in resp.iter_lines():
if line.startswith("data: "):
print(line[6:]) # SSE payloadWebhook Events
Webhook Event Flow
Webhook Payload Schemas
All webhook payloads share a common envelope. Verify the signature using X-G8KEPR-Signature (HMAC-SHA256 of the raw body with your webhook secret).
{
"event": "threat.blocked",
"version": "2.1",
"timestamp": "2026-03-31T14:22:01.432Z",
"org_id": "org_abc123",
"gateway_id": "gw_prod_01",
"data": {
"request_id": "req_xyz789",
"threat_score": 0.91,
"decision": "block",
"pattern_ids": ["PI-001", "PI-007"],
"session_id": "sess_def456",
"session_request_count": 4,
"session_escalation_factor": 2.0,
"source_ip": "1.2.3.4",
"method": "POST",
"path": "/v1/completions",
"user_agent": "python-httpx/0.27.0"
}
}{
"event": "gateway.down",
"version": "2.1",
"timestamp": "2026-03-31T14:30:00.001Z",
"org_id": "org_abc123",
"gateway_id": "gw_prod_01",
"data": {
"gateway_name": "Production",
"backend_url": "https://api.example.com",
"check_type": "http",
"http_status": 503,
"consecutive_failures": 3,
"first_failure_at": "2026-03-31T14:29:00.000Z"
}
}{
"event": "rate_limit.exceeded",
"version": "2.1",
"timestamp": "2026-03-31T15:00:05.210Z",
"org_id": "org_abc123",
"gateway_id": "gw_prod_01",
"data": {
"scope": "per_ip",
"identifier": "1.2.3.4",
"limit": 100,
"window_seconds": 60,
"retry_after": 47
}
}Signature verification
X-G8KEPR-Signature before processing webhook payloads. Use hmac.compare_digest (Python) or crypto.timingSafeEqual (Node) to prevent timing attacks.Threat Detection
Threat Detection Pipeline
Request received
1,500+ signatures
Behavioral detection
Rules evaluation
Allow / Block
1,500+
Threat patterns
15
Attack categories
<2ms
Avg analysis latency
Session Escalation Tiers
1 request
1.0×
Baseline score
3 requests
1.3×
Elevated threshold
5 requests
2.0×
High alert
8+ requests
2.5×
Auto-block
Live Threat Log
Response Policies
| Policy | Threshold | Action | Description |
|---|---|---|---|
| Log | 0.2 – 0.4 | Allow + log | Record event for analysis. No impact to request. |
| Alert | 0.4 – 0.6 | Allow + alert | Webhook notification sent. Request continues. |
| Challenge | 0.6 – 0.7 | CAPTCHA / 2FA | User must verify before request is forwarded. |
| Block | ≥ 0.7 | HTTP 403 | Request rejected. Audit log written. |
Auto-block threshold
POST /v1/config/thresholds. Lowering it increases false-positive rate; raise it only after reviewing your threat logs.CLI Reference
Installation
brew tap g8kepr/tap
brew install g8kepr
g8kepr --version
# g8kepr v2.1.0Authentication
# Login with browser OAuth
g8kepr login
# Or supply API key directly
g8kepr login --api-key YOUR_API_KEY
# Verify auth status
g8kepr auth status
# Logged in as: wesley@example.com (org: acme-corp)Command Reference
| Command | Flags | Description |
|---|---|---|
| g8kepr login | --api-key, --org | Authenticate with G8KEPR |
| g8kepr auth status | — | Show current auth context |
| g8kepr gateway list | --json, --org | List all gateways |
| g8kepr gateway create | --name, --backend-url | Create a new gateway |
| g8kepr gateway delete | --id, --force | Delete a gateway |
| g8kepr threats stream | --gateway, --since | Stream live threat events |
| g8kepr threats export | --start, --end, --format | Export threat log to CSV/JSON |
| g8kepr rules add | --type, --config | Add rate-limit or geo-blocking rule |
| g8kepr rules list | --gateway, --type | List active rules |
| g8kepr rules delete | --id | Delete a rule |
| g8kepr config set | --key, --value | Update a config value |
| g8kepr config show | --json | Show current configuration |
| g8kepr health | --verbose | Check platform health |
| g8kepr logs | --gateway, --level | Tail platform logs |
| g8kepr api-keys list | --org | List API keys for org |
Machine-readable output
--json for machine-readable output. Pipe to jq in CI: g8kepr threats export --format json | jq '.[] | select(.score > 0.7)'API Reference
API Key Authentication Flow
Authorization: Bearerscopes, limitsg8k_live_*Production - Full accessg8k_test_*Sandbox - No billingg8k_rstr_*Restricted - Limitedhttps://api.g8kepr.com/v1Authentication
# All requests require Bearer token
curl https://api.g8kepr.com/v1/gateways \
-H "Authorization: Bearer YOUR_API_KEY"
# Rotate a key (old key remains valid for 15 min)
curl -X POST https://api.g8kepr.com/v1/api-keys/rotate \
-H "Authorization: Bearer YOUR_API_KEY" \
-d '{ "key_id": "key_abc123" }'Endpoints
Rate Limit Response Headers
| Header | Description |
|---|---|
| X-RateLimit-Limit | Maximum requests allowed in the current window |
| X-RateLimit-Remaining | Requests remaining in current window |
| X-RateLimit-Reset | Unix timestamp when the window resets |
| Retry-After | Seconds to wait before retrying (on 429 only) |
Error codes
Retry-After header. 503 Service Unavailable — Redis or DB health check failed; retry with backoff.Configuration
Configuration File Structure
g8kepr.yaml.envCompliance Frameworks
Compliance Framework Coverage
Environment Variables
| Variable | Required | Default | Description |
|---|---|---|---|
| DATABASE_URL | required | — | PostgreSQL connection string |
| REDIS_URL | required | — | Redis connection string for rate limiting & sessions |
| SECRET_KEY | required | — | 32+ char secret for JWT signing |
| API_ENCRYPTION_KEY | required | — | Fernet key for encrypting stored API keys |
| ENVIRONMENT | optional | production | production | staging | development |
| LOG_LEVEL | optional | info | debug | info | warning | error |
| OPENAI_API_KEY | optional | — | Required to use OpenAI via AI Gateway |
| ANTHROPIC_API_KEY | optional | — | Required to use Anthropic via AI Gateway |
| SENTRY_DSN | optional | — | Error monitoring (recommended for production) |
| POSTHOG_API_KEY | optional | — | Analytics (PII-safe — no user content logged) |
| ALLOWED_ORIGINS | optional | * | Comma-separated CORS allowed origins |
| WORKERS | optional | 4 | Uvicorn worker count (set to CPU cores × 2) |
Docker Compose — env injection
services:
backend:
image: g8kepr/api:2.1.0
environment:
DATABASE_URL: ${DATABASE_URL}
REDIS_URL: ${REDIS_URL}
SECRET_KEY: ${SECRET_KEY}
API_ENCRYPTION_KEY: ${API_ENCRYPTION_KEY}
ENVIRONMENT: production
LOG_LEVEL: info
WORKERS: 4
env_file:
- .env.production # blocked from git by pre-commit hookFeature Flags
| Flag | Default | Effect when enabled |
|---|---|---|
| FEATURE_MCP_SECURITY | true | Enable MCP Security pillar and session tracking |
| FEATURE_AI_GATEWAY | true | Enable AI Gateway routing and marketplace |
| FEATURE_GEO_BLOCKING | false | Enable geo-blocking rules UI and enforcement |
| FEATURE_HIPAA_BAA | false | Show HIPAA BAA module (requires enterprise plan) |
| FEATURE_SOC2_REPORTS | false | Enable SOC 2 compliance reporting tab |
Secrets management
.env.production commits are blocked by the pre-commit hook. Use your cloud provider's secrets manager (AWS Secrets Manager, GCP Secret Manager, Vault) and inject at runtime via env_file or environment variable injection.Troubleshooting
Common Errors
Debug Mode
# Enable verbose logging
LOG_LEVEL=debug g8kepr serve
# Or set in environment
export LOG_LEVEL=debug
docker compose up backendHealth Check Endpoints
| Endpoint | Checks | Use case |
|---|---|---|
| GET /health | Process alive | Load balancer liveness probe |
| GET /health/db | PostgreSQL connectivity + query | Readiness probe, alerting |
| GET /health/redis | Redis PING + latency | Readiness probe, alerting |
| GET /health/full | All of the above + worker queue | Monitoring dashboards |
Enterprise Support
Priority support, SLA, and dedicated Slack channel for Enterprise plans.
Contact sales →Monitoring & Observability
G8KEPR ships with a full observability stack: Prometheus metrics, pre-built Grafana dashboards, and structured JSON logs compatible with any aggregator (Loki, Datadog, ELK).
Prometheus Metrics
Metrics are exposed at GET /metrics in Prometheus text format. Scrape interval recommended: 15s.
| Metric | Type | Description |
|---|---|---|
| g8kepr_requests_total | Counter | Total requests proxied, labeled by gateway, method, status |
| g8kepr_request_duration_seconds | Histogram | End-to-end request latency (p50/p95/p99 available) |
| g8kepr_threats_total | Counter | Threats detected, labeled by decision (block/alert/log) |
| g8kepr_threat_score | Histogram | Distribution of threat scores across all requests |
| g8kepr_rate_limit_hits_total | Counter | Rate limit 429 responses, labeled by scope and gateway |
| g8kepr_session_escalations_total | Counter | Multi-turn session escalation events by tier |
| g8kepr_gateway_health | Gauge | 1=healthy 0=unhealthy, labeled by gateway_id |
| g8kepr_db_pool_size | Gauge | Active / idle PostgreSQL connections |
| g8kepr_redis_latency_seconds | Histogram | Redis command latency |
| g8kepr_ai_tokens_total | Counter | LLM tokens consumed via AI Gateway, labeled by model and provider |
Prometheus Scrape Config
scrape_configs:
- job_name: 'g8kepr'
scrape_interval: 15s
static_configs:
- targets: ['g8kepr-backend:8000']
metrics_path: /metrics
bearer_token: YOUR_METRICS_TOKENGrafana Dashboard Setup
Import the pre-built dashboards from the monitoring/grafana/dashboards/ directory in the repo. Three dashboards are included:
G8KEPR Overview
Request volume, threat rate, gateway health, p95 latency — the oncall dashboard
Threat Intelligence
Threat score distribution, top patterns, session escalations, geo heatmap
AI Gateway
Token consumption, cost by model/provider, routing distribution, error rates
# Import via Grafana CLI
grafana-cli dashboards import monitoring/grafana/dashboards/overview.json
grafana-cli dashboards import monitoring/grafana/dashboards/threats.json
grafana-cli dashboards import monitoring/grafana/dashboards/ai-gateway.json
# Or via API
curl -X POST http://grafana:3000/api/dashboards/import \
-H "Authorization: Basic admin:admin" \
-H "Content-Type: application/json" \
-d @monitoring/grafana/dashboards/overview.jsonAlert Rules
| Alert | Severity | Condition | Default threshold |
|---|---|---|---|
| GatewayDown | critical | g8kepr_gateway_health == 0 | Immediate |
| HighThreatRate | warning | rate(g8kepr_threats_total[5m]) > N | > 50/min |
| HighLatency | warning | p95 request duration | > 2s for 5 min |
| ContainerRestarting | critical | restart count delta | > 3 in 15 min |
| RedisLatencyHigh | warning | g8kepr_redis_latency_seconds p99 | > 100ms |
| TokenBudgetNearing | warning | daily token spend vs limit | > 80% of budget |
Log Format
All logs are structured JSON, written to stdout. Each log line includes request_id for end-to-end tracing.
{
"level": "info",
"time": "2026-03-31T14:22:01.432Z",
"request_id": "req_xyz789",
"org_id": "org_abc123",
"gateway_id": "gw_prod_01",
"method": "POST",
"path": "/v1/completions",
"status": 403,
"duration_ms": 4,
"threat_score": 0.91,
"decision": "block",
"pattern": "PI-001"
}Changelog
All notable changes to G8KEPR. We follow Semantic Versioning.
- NewMCP Security pillar — session tracking, tool RBAC, explainability scoring
- NewMulti-turn session correlation with 1.3× / 2.0× / 2.5× escalation tiers
- NewIdempotency-Key middleware (Redis-backed, 24h TTL) on all write endpoints
- ImprovedAPI Security raised to A+ — all 12 Core Standards satisfied
- ImprovedRate limiter moved from in-process Map to nginx limit_req_zone (multi-worker safe)
- FixedWebSocket IDOR — gateway authorization now runs before websocket.accept()
- FixedGateway proxy fail-open on Redis error changed to 503 + Retry-After
- NewAI Gateway pillar — multi-provider routing with 4 strategies (round-robin, least-latency, cost-optimized, failover)
- NewModel marketplace supporting OpenAI, Anthropic, Cohere, Mistral, and Google
- NewKubernetes Helm chart v1.0.0 with HPA, PodDisruptionBudget, and zone-aware scheduling
- ImprovedAuth upgraded to A+ — full RBAC matrix, billing:manage permission, MFA for destructive ops
- ImprovedMulti-tenant isolation verified: PostgreSQL RLS + app-layer org_id + Redis key-scoping
- BreakingAPI v0 endpoints removed. Migrate to /v1/ — see migration guide below.
- NewPatroni HA mode — automatic PostgreSQL failover with <30s RTO
- NewPrometheus metrics endpoint + 3 pre-built Grafana dashboards
- NewWebhook system — threat.blocked, gateway.down, rate_limit.exceeded events
- ImprovedDocker images fully pinned with SHA digests — no more :latest tag surprises
- FixedBFF SQL injection on filter column names — now validated via allowlist
- NewEU AI Act compliance module with Art. 9/12/13 evidence generation
- NewSOC 2 Type I audit trail — tamper-evident audit log with 7-year retention
- NewGeo-blocking rules with country-level allow/deny lists
- ImprovedThreat detection patterns expanded from 682 → 1,500+ across 15 categories
- ImprovedBackend test coverage raised from 0% → 31.70% (gate: 70%, module-scope: 52.65%)
Start protecting your APIs today
Free tier includes 10,000 requests/day, full threat detection, and all four security pillars. No credit card required.