AEGIS: Building a PII Data Privacy Vault from Scratch

Table of contents

The Problem

I’ve done it too. Pasted an API key, a user’s email, a token into ChatGPT or Claude to debug something faster. You look up and realize everyone around you is doing the same thing.

That’s just the visible half of the problem. The quieter half: most of those same systems store PII as plain text in a database. email column in the users table. card_number in payments. phone in orders. Sitting there, unmasked, accessible to anyone with a read query.

Data breaches keep making headlines. Most of them trace back to this — sensitive data sitting exposed, one misconfigured query or leaked dump away from being someone’s problem. A misconfigured analytics service, an over-privileged intern with prod DB access, a dump that ends up on Pastebin — and suddenly raw PII is everywhere it shouldn’t be.

The standard advice is “encrypt sensitive columns.” But column-level encryption at rest doesn’t help when your ORM decrypts everything on read. Your analytics team can still SELECT email FROM users. Your logs can still contain card numbers. The data is accessible to your entire application layer, which means it’s only as safe as your least-secure service.

The real fix is isolation. Sensitive data should live in a separate system with its own access controls, its own encryption keys, and its own audit trail. Your main application never stores the raw value — it stores a token that references it.

That’s AEGIS.

What AEGIS Does

AEGIS is a REST API vault for PII data. The core idea:

1. Your app sends { "field_type": "email", "value": "john@example.com" } to AEGIS
2. AEGIS stores the encrypted value, returns a token: tok_7f3a9c1e-4b2d-...
3. Your app stores the token, never the raw value
4. When something needs the real email, it calls AEGIS with the token
5. What it gets back depends on the caller’s role

Application DB:
  user_id | email_token
  --------+------------------------------------
  1       | tok_7f3a9c1e-4b2d-4f8a-9c1e-...

AEGIS Vault DB:
  token               | enc_value (AES-256-GCM) | nonce
  --------------------+-------------------------+------
  tok_7f3a9c1e-...   | dGhpcyBpcyBjaXBoZXJ0... | cmFuZG9t...

Your main DB never has a plaintext PII value. Even if it’s fully compromised, there’s nothing to read.

Architecture

The system has four main components:

Vault — tokenize and detokenize PII, field-level access control per role

Auth — JWT-based authentication, RBAC with four roles

Audit — append-only log of every access, including denied ones

LLM Proxy — tokenizes PII in prompts before sending to OpenAI, detokenizes responses

Encryption: AES-256-GCM

Every value stored in the vault is encrypted with AES-256-GCM using a master key from the environment.

func Encrypt(plaintext string, key []byte) (ciphertext, nonce string, err error) {
    block, err := aes.NewCipher(key)
    gcm, err := cipher.NewGCM(block)

    nonceBytes := make([]byte, gcm.NonceSize())
    io.ReadFull(rand.Reader, nonceBytes)  // fresh random nonce per record

    encrypted := gcm.Seal(nil, nonceBytes, []byte(plaintext), nil)
    return base64.StdEncoding.EncodeToString(encrypted),
        base64.StdEncoding.EncodeToString(nonceBytes),
        nil
}

Two decisions worth calling out:

Why GCM? AES-256-GCM is authenticated encryption — it provides confidentiality and integrity. If anyone tampers with the ciphertext in the database, decryption fails with an authentication tag mismatch rather than silently returning garbage. CBC mode doesn’t give you this.

Why a fresh nonce per record? If you reuse a nonce with the same key, GCM’s security collapses completely — two ciphertexts encrypted with the same key+nonce can be XORed to cancel out the keystream. A 12-byte random nonce per record means identical PII values produce different ciphertexts each time, so there’s no pattern to exploit even if the attacker sees the full database.

The vault schema stores both:

CREATE TABLE vault_records (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    token       VARCHAR(100) UNIQUE NOT NULL,
    field_type  VARCHAR(50) NOT NULL,
    enc_value   TEXT NOT NULL,      -- base64 AES-256-GCM ciphertext
    nonce       VARCHAR(100) NOT NULL,  -- base64 12-byte random nonce
    created_by  UUID REFERENCES users(id),
    created_at  TIMESTAMPTZ DEFAULT NOW(),
    deleted_at  TIMESTAMPTZ          -- soft delete, not hard
);

deleted_at is a soft delete. Hard-deleting vault records would leave dangling references in the audit log — every audit entry references a token, and that audit trail is the compliance record. Soft delete makes the record inaccessible through the API while preserving the audit reference.

Role-Based Access: Field-Level Granularity

Not everyone who needs to access PII needs the same level of access. A payment processor needs card numbers. An analyst running reports doesn’t. A customer support rep needs enough to identify a caller, not their full financial data.

AEGIS has four roles with field-level access control:

ANALYST can see masked emails but is explicitly denied card numbers — not masked, denied. SERVICE can retrieve full card numbers (payment processing) but only sees masked versions of everything else.

The access matrix in the vault service:

var accessMatrix = map[string]map[string]string{
    "ADMIN": {
        "": "FULL",  // default: FULL for any field
    },
    "ANALYST": {
        "email":       "MASKED",
        "name":        "MASKED",
        "card_number": "DENIED",  // explicit deny
        "":            "MASKED",  // default: MASKED
    },
    "SERVICE": {
        "card_number": "FULL",    // payment processing
        "":            "MASKED",  // default: MASKED
    },
    "VIEWER": {
        "": "MASKED",  // default: MASKED for everything
    },
}

The "" key is the default fallback — any field type not explicitly listed falls back to the role’s default. This means adding a new field type (e.g., passport) automatically gets the safest default for each role without touching the access matrix.

The detokenize flow:

func (s *Service) Detokenize(token, role string) (value, accessLevel string, err error) {
    record, _ := s.repo.FindByToken(token)

    accessLevel = resolveAccess(role, record.FieldType)
    if accessLevel == "DENIED" {
        return "", "DENIED", errors.New("access denied for this field type")
    }

    plaintext, _ := crypto.Decrypt(record.EncValue, record.Nonce, key)

    if accessLevel == "MASKED" {
        plaintext = crypto.MaskValue(plaintext, record.FieldType)
    }
    return plaintext, accessLevel, nil
}

Role is embedded in the JWT, so there’s no DB lookup on every request. The tradeoff: role changes only take effect on next login. Short JWT expiry (JWT_EXPIRY_HOURS) keeps the stale-role window small.

Field-Specific Masking

MASKED doesn’t mean the same thing for every field type. The masking logic is field-aware:

email:   john@example.com  →  j***@example.com
phone:   9876543210        →  ******3210
card:    4111111111114242  →  ****-****-****-4242
aadhaar: 123456781234      →  XXXX-XXXX-1234
pan:     ABCDE1234F        →  ABCDE****F
name:    John Doe          →  J*** D***
dob:     1990-07-15        →  ****-**-15

Each format preserves just enough for identification (last 4 digits, first initial, day of birth) while hiding the actionable data. An analyst can confirm “yes, this is the account for J_ D_” without getting a full name they could misuse.

Append-Only Audit Logs

Every vault operation is logged — tokenize, detokenize, delete — including failed attempts and denied accesses.

CREATE TABLE audit_logs (
    id             UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    actor_id       UUID REFERENCES users(id),
    action         VARCHAR(50) NOT NULL,  -- STORE, DETOKENIZE, DELETE, LLM_PROXY
    token          VARCHAR(100),
    field_type     VARCHAR(50),
    access_level   VARCHAR(20),           -- FULL, MASKED, DENIED
    ip_address     VARCHAR(45),
    success        BOOLEAN NOT NULL,
    failure_reason TEXT,
    created_at     TIMESTAMPTZ DEFAULT NOW()
);

There’s no update or delete endpoint for audit records. The audit trail’s value is its immutability — if someone queries denied fields repeatedly from an unusual IP, that pattern is permanently recorded.

A DENIED access is logged just like a successful one. This matters: an attacker probing the API to find exploitable endpoints will leave a trail even if every request fails.

The LLM Proxy: Stopping PII at the AI Boundary

This was the original spark. Watching teams paste user data into ChatGPT to debug production issues — emails, tokens, sometimes full user records. It’s fast, it’s tempting, and it puts PII directly into an external model’s request logs and potentially its training pipeline.

As LLMs get embedded in more applications, the same leak moves into the product itself. You’re building a customer support assistant. A user asks: “Can you look up my account for email john@example.com?” Your app passes this prompt to OpenAI. Now john@example.com is in OpenAI’s request logs, potentially in training data, definitely outside your compliance boundary.

The naive fix is to mask PII before sending: replace john@example.com with j***@example.com. But now the LLM can’t use the actual email in its response, can’t reference it in a reply, can’t take action on it.

AEGIS solves this with round-trip tokenization:

User:  "email john@example.com about order 12345"
           ↓ detector.go scans for PII regex patterns
           ↓ vault.Tokenize("email", "john@example.com")
Prompt to LLM: "email tok_abc123... about order 12345"
           ↓ LLM responds with token in output
LLM:   "I'll send a confirmation to tok_abc123..."
           ↓ vault.Detokenize("tok_abc123...", "ADMIN")
User:  "I'll send a confirmation to john@example.com"

The LLM operates on stable token references. The model never sees raw PII. The caller gets human-readable output with real values restored.

Every proxy call creates an LLM_PROXY audit entry recording which tokens were exposed to which model, by which actor. This is the compliance record for AI data handling — auditable evidence that PII wasn’t sent to an external model in plaintext.

Detection is regex-based, no external NLP:

One important fail-safe: if tokenization fails for any detected PII value, the proxy returns a 500 and does not forward the prompt. Raw PII is never sent to an external model even in error paths.

Threat Model

The right question for any vault system: what happens if it’s compromised?

Database leaked (no master key)

The attacker gets rows of enc_value + nonce — both base64-encoded AES-256-GCM ciphertext. Without VAULT_MASTER_KEY, these are computationally infeasible to reverse. A random nonce per record means there’s no pattern across ciphertexts. Raw PII is safe.

VAULT_MASTER_KEY leaked

This is the single critical secret. If it leaks alongside the database dump, every record can be decrypted. This is game over for stored data.

Production mitigation:

• Never store in a .env file on the server
• Use AWS KMS, HashiCorp Vault, or GCP Secret Manager
• Ideally: envelope encryption — the master key itself is encrypted by a hardware-backed KMS key that never leaves the HSM
• For maximum isolation: Customer-Managed Keys (CMK) where the key encryption key (KEK) lives in the customer’s own AWS KMS account — a full vendor-side compromise still can’t decrypt customer data

JWT_SECRET leaked

An attacker can forge valid JWTs with ADMIN role and call /detokenize for any token. Mitigation: short JWT_EXPIRY_HOURS, immediate rotation on suspicion, use RS256 (asymmetric) JWTs in production so the signing key never needs to be distributed.

Audit logs as detection

Every detokenize call is logged with actor_id, ip_address, and timestamp. High volume from a single actor, unusual IPs, DENIED spikes — these patterns signal an active probe before full exfiltration completes. The audit trail is both compliance record and early warning system.

Running It

cp .env.example .env
# Set JWT_SECRET (32+ chars) and VAULT_MASTER_KEY (exactly 32 chars)

docker-compose up --build
# Postgres starts, migrations run, API live at http://localhost:8080

# Register and login
curl -X POST http://localhost:8080/api/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{"username": "alice", "password": "password123", "role": "ADMIN"}'

TOKEN=$(curl -s -X POST http://localhost:8080/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"alice","password":"password123"}' | jq -r '.data.token')

# Tokenize PII
curl -X POST http://localhost:8080/api/v1/vault/tokenize \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"field_type": "email", "value": "john@example.com"}'
# Returns: {"data": {"token": "tok_7f3a9c1e-..."}}

# Detokenize — response depends on your role
curl -X POST http://localhost:8080/api/v1/vault/detokenize \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"token": "tok_7f3a9c1e-..."}'

What’s Next

The LLM proxy is the area with the most interesting problems left to solve:

Contextual PII detection — regex catches structured formats (email, PAN, card numbers) but misses unstructured PII like "the user's name is John Doe". A lightweight NER model running locally could catch what regex misses without adding external dependencies.

Streaming responses — current implementation buffers the full LLM response before detokenizing. For long responses this adds latency. Streaming detokenization requires scanning the token stream as it arrives, which is a harder problem.

Key rotation — VAULT_MASTER_KEY rotation requires re-encrypting every record in the vault. Envelope encryption (data keys encrypted by a master key) would make rotation cheaper — rotate the master key, re-encrypt only the data keys, not the vault records.

The code is on GitHub: github.com/anandukch/Aegis