Empirical Benchmark: 9,334,805 Prompts Evaluated with 100.00% Roundtrip Fidelity & 86µs Latency
A practical, empirical audit for high-throughput sovereign AI privacy: evaluating zero information loss, sub-millisecond edge latency, and enterprise SLA stability across 1.94 billion tokens.
Luckily, edge-native de-identification breaks the tradition of slow, lossy NLP anonymization pipelines. Traditional Python-based solutions like Microsoft Presidio or spaCy require heavy compute instances and typically incur 50ms to 300ms of overhead per request.
To validate whether an edge-native privacy engine can sustain enterprise production workloads with zero character distortion, we executed an exhaustive empirical audit over a composite corpus of 9,334,805 prompts (~1.94 billion tokens) and 10,281,399 protected entities.
The Unified Multi-Source Dataset
Evaluating privacy tokenizers only on synthetic dummy strings is insufficient. Real-world prompts contain complex punctuation, Unicode emojis, code blocks, tracking numbers, and conversational slang. Our benchmark corpus combined five premier datasets into 94 Apache Parquet row groups:
- LMSYS Chatbot Arena: 2.52M real multi-turn interactions with frontier LLMs featuring natural conversational phrasing and multilingual queries.
- OpenOrca: 2.91M complex technical instructions, Python stack traces, UUIDs, mathematical expressions, and SQL statements.
- WildChat: 2.15M unfiltered global user prompts covering edge Unicode delimiters and non-Latin scripts.
- Enron Email Corpus: 1.02M corporate email headers, signatures, direct phone lines, work emails, and executive names.
- Customer Support Twitter: 734K unstructured complaints containing courier tracking numbers (UPS, FedEx) and invoice IDs.
- Sovereign ID Synthesis: Algorithmic test vectors across 109 jurisdictions (Aadhaar, SSN, NRIC, Codice Fiscale, and Luhn-valid cards).
Full In-Memory Engine Audit (9,334,805 Prompts)
In this stage, the standalone engine executed on an 8-worker thread pool. The goal was to test pure V8 computation speed, memory stability, and zero-collision determinism.
| Benchmark Metric | Observed Value | Auditable Standard |
|---|---|---|
| Total Prompts Evaluated | 9,334,805 | 100% of entire 94-row-group Parquet corpus |
| Exact Roundtrip Matches | 9,334,805 | 100.00% Bit-for-Bit Exact Equality |
| Roundtrip Mismatches | 0 (Zero) | Zero token collisions or dropped characters |
| Total Entities Protected | 10,281,399 | Sovereign IDs, cards, emails, phones, tracking IDs |
| Estimated Tokens Analyzed | 1,937,516,961 | ~1.94 Billion tokens of text processed end-to-end |
| Total Execution Time | 526.35 s (8.77 min) | Continuous multi-threaded execution |
| Overall Throughput | 17,735 prompts / sec | Sustained engine throughput under maximum load |
| Average Processing Latency | 0.314 ms (314 µs) | Sub-millisecond processing per prompt |
| Median Latency (p50) | 0.086 ms (86 µs) | Ultra-low compute footprint |
50,000 Prompts HTTPS Socket SLA Audit
In-memory benchmarks prove engine speed; network benchmarks prove real-world production SLAs. We ran 50,000 real-world prompts uniformly sampled across all row groups over TLS 1.3 encrypted HTTPS connections with a 30-stream socket pool.
| Network Metric | Observed Performance | Operational Notes |
|---|---|---|
| Sampled Prompts | 50,000 prompts | Uniformly sampled across 9.33M dataset |
| Total HTTPS Network Requests | 100,000 requests | 50k Tokenize + 50k Detokenize calls over TLS 1.3 |
| Elapsed Network Time | 37.15 seconds | 30 concurrent socket streams |
| Overall Network RPS | 2,691.95 HTTP req / sec | Continuous TLS 1.3 socket throughput |
| Network Prompt Throughput | 1,345.98 prompts / sec | End-to-end client-to-API processing |
| Token Throughput | 277,506 tokens / sec | Estimated at standard 4-char token density |
| Socket Drop / Timeout Errors | 0 (Zero) | 100% connection reliability |
Real-World Latency Percentile Matrix
The table below details response times across percentiles under sustained 30-stream concurrent production load over TLS 1.3 sockets:
| Operation | Min | p50 (Median) | p90 | p95 | p99 | Max |
|---|---|---|---|---|---|---|
| POST /v1/tokenize | 4.60 ms | 13.30 ms | 20.98 ms | 25.22 ms | 38.00 ms | 73.93 ms |
| POST /v1/detokenize | 4.50 ms | 7.20 ms | 10.91 ms | 12.81 ms | 18.31 ms | 64.22 ms |
| Total Roundtrip | 12.06 ms | 20.81 ms | 30.86 ms | 36.36 ms | 51.55 ms | 104.20 ms |
Evolution to Absolute 100.00% Perfection
Achieving 100% roundtrip fidelity on nearly 10 million prompts is rare in data engineering. Across three successive benchmark iterations, we systematically eliminated edge-case collisions:
(?<![A-Za-z0-9]) to phone patterns and added universal boundary caps, resolving 330 of the 339 mismatches.
Ford-150) colliding with invoice patterns, and Colombian national ID (CC992140300) prefix grouping. Once boundary checks were applied, all 9,334,805 prompts achieved bit-for-bit exact roundtrip equality.
Live Edge-Case Verifier
Try tokenizing and restoring one of the historically challenging edge cases in real-time below:
Reproduce the Benchmarks Locally
All benchmarks are 100% auditable and reproducible using Node.js 18+ and the open-source repository:
# 1. Clone repository and install dependencies git clone https://github.com/PriyanujBoruah/AI-Privacy-Core.git cd AI-Privacy-Core npm install # 2. Run automated test suite (102 passing tests) npm test # 3. Execute 50,000-prompt HTTPS socket benchmark node scripts/http_latency_benchmark.mjs --samples 50000 --concurrency 30 # 4. Execute direct multi-core engine audit on unified Parquet corpus node scripts/multicore_benchmark.mjs --workers 8
We understand that this is an empirical overview, but at its core we've seen this architecture unlock production LLM deployments without compromising privacy or incurring perceptible latency penalties.
If you're evaluating a sovereign privacy layer for your enterprise or building multi-tenant AI pipelines, we encourage you to follow future benchmark blogs, test the playground, or reach out to support@projectspg.info. And we'd love any feedback on what may be missing here. Happy building!