Arch Introduction

Arch operates by handling undifferentiated tasks in prompt processing through several key mechanisms. It uses specialized LLMs for detecting and rejecting jailbreak attempts, intelligently calling backend APIs to fulfill user requests, and managing prompt routing between upstream LLMs. The system runs alongside application servers and leverages Envoy's HTTP management capabilities for handling ingress and egress traffic. It includes purpose-built LLMs for function/API calling and parameter extraction, while centralizing prompt guardrails through its Prompt Guard feature. The platform implements the W3C Trace Context standard for comprehensive request tracing and provides robust traffic management with features like smart retries and automatic cutover.