beops
all projects

aissist

incident summarizer shipping real customer text through azure openai — with a tiered pii guard at the chokepoint and a self-serve secure-llm-use pattern that replaced per-team security review.
draft aipythonazuresecurity
year
2025
role
engineer
stack
python · azure data explorer · azure openai · mem0 · presidio
link

automates the dull part of incident management. three-stage pipeline: fetch data from azure data explorer (kusto), transform it into structured json, then run it through azure openai to produce summaries, escalation notes, troubleshooting guides.

what it does

  • persistent azure auth — authenticate once per token lifetime
  • screenshot processing — downloads and processes embedded screenshots from incident data
  • memory integration via mem0 — learns from previous incidents across sessions
  • gap analysis — compares incident troubleshooting against knowledge base articles to find missing steps
  • multi-incident support — processes single incidents or combines related ones for unified analysis

pipeline

  1. kusto_fetcher — retrieves raw incident data, extracts screenshots, organizes by incident folder
  2. transformer — csv to structured json optimized for ai processing
  3. summarizer — generates escalation notes, mitigation reports, troubleshooting guides via azure openai

the guard layer

shipping real incident text to a model provider is a data problem before it is a quality problem, so the app grew a security layer (guard/):

  • input filtering — tiered redaction (regex always on, presidio ner, opt-in llm detector) wrapped around the single client constructor every azure openai client is built from, so all call sites are covered by editing one file. responses are rehydrated so downstream consumers still see real values.
  • output validation — validates llm-generated content at the write boundary (the azure devops path) and runs after rehydration, because a model emitting a placeholder in the wrong place reintroduces a real value.
  • provider-tier data classification — a four-class matrix (public / internal / confidential / restricted) mapped to endpoint tiers, declared per deployment and enforced at client construction: a forbidden payload/endpoint pair fails at startup, not at the first customer payload.
  • self-serve security review — the whole layer is packaged as a pattern doc plus a pr template (sections for data class, filtering, validation, and a ten-question threat-model shortcut), cutting per-team review from a multi-week engagement to something teams complete themselves.