About ChangeProof Agent

tl;dr

• ChangeProof Agent is an AI agent that generates risk assessments, approval briefs, execution plans, rollback plans, post-change reports, and Evidence Bundles for DNS change work.
• The point is not to let AI change DNS records. The point is to prepare the judgment material and evidence before and after a human does the work.
• I built it with Azure OpenAI, Semantic Kernel, ASP.NET Core, React, SQLite, Azure Blob Storage, and Azure App Service.
• The agent pipeline is split into Change Intake, DNS Risk Assessment, Approval Brief, Execution Plan, Rollback Plan, and Evidence Report.
• It does not guarantee audit compliance, but I built it as an MVP to reduce the “we cannot explain this later” problem in small IT change work.

What I Built

• App name: ChangeProof Agent
• URL: not publicly available
• Video: demo video
• Stack: Azure App Service / Azure OpenAI / Microsoft Foundry / Semantic Kernel / ASP.NET Core 8 / React / TypeScript / SQLite / Azure Blob Storage / Application Insights
• Target workflow: DNS change risk review, approval text generation, execution plan generation, rollback plan generation, post-change report draft generation, and Evidence Bundle storage

Target Users, Problem, and Solution

The target users are small IT companies, web production companies, solo internal IT people, and small teams that manage customer sites or internal systems.

For this group, small change work such as DNS changes and server cutovers often does not go through a heavy ticketing or change management system. The person doing the work may understand what is happening at the time, but later it can become unclear what the previous state was, who approved the change, and how the team planned to roll back if something failed.

With ChangeProof Agent, you enter a DNS change request, and the AI generates risks, unknowns, an approval brief, an execution plan, a rollback plan, and a post-change report draft. It then stores them as an Evidence Bundle. I built it as a tool for making DNS changes closer to an explainable business process.

Why DNS Changes?

I think small IT changes are exactly where evidence tends to become vague. Since the hackathon time was limited, I tried to keep the development scope as narrow as possible.

Large production releases usually have release review meetings and runbooks. On the other hand, work such as DNS changes, server cutovers, and mail-related record updates often happens inside the head of an experienced operator.

The hard part is not the DNS change itself.

• The previous state is not recorded
• Customer approval is ambiguous
• The reason for saying “OK” is not recorded
• There is no rollback procedure
• The post-change report is written after the fact

These become problems during incidents, audits, and handovers.

From an ISMS point of view, I think this area matters more than it first appears. The issue is not only change management itself. If you cannot explain who approved what, based on which information, for which scope of change, and what was checked after the work, you get stuck later. That is painful not only during incidents, but also during audits and retrospectives.

So this time, I placed the AI agent’s role around preparation, explanation, and recording.

What the MVP Does

I built an MVP called ChangeProof Agent for Microsoft Agent Hackathon 2026.

The scope is limited to DNS changes. For example, changing an A record from an old IP address to a new IP address for a website cutover, lowering TTL, and checking possible impact on MX / TXT / SPF / DKIM / DMARC records.

The flow is simple. Load a sample change request, create a Change Request, and press Analyze. Then the AI generates the following:

• Impact scope
• Risks
• Pre-check items
• Approval request text for the customer
• Execution steps for the human operator
• Rollback steps
• Post-change report draft
• Evidence Bundle

The screen layout is also simple: Change Request list on the left, JSON input in the center, and AI output tabs on the right. I did not aim for a flashy UI. I prioritized making the demo flow easy to follow.

What This MVP Does Not Do

ChangeProof Agent does not automatically execute DNS changes.

This MVP intentionally does not do the following:

• Change real DNS records
• Automatically apply changes to production
• Let AI make approval decisions
• Guarantee audit compliance
• Guarantee legal evidential value

By drawing this line clearly, I limited the AI’s role to work support and record creation.

Demo Flow

1. Use Load Sample to load a sample DNS change
2. Use Create to create a Change Request
3. Press Analyze to run the agent pipeline for the registered change request
4. Review the Risk Assessment
5. Review the Approval Brief
6. Review the Execution Plan and Rollback Plan
7. Review the Evidence Report and Audit Events
8. Download the Evidence Bundle as JSON / Markdown

In the sample, the current A record is 198.51.100.20 TTL 3600, and the target state is 203.0.113.10 TTL 300. I expected the AI to point out TTL, website cutover, and unknowns around mail-related records.

The important point is that the AI should not simply declare that the change is safe. It should separate what can be known from the input, what it assumes, and what is still unknown.

Architecture

I kept the architecture fairly ordinary.

Browser
  |
  v
Azure App Service
  |
  +--> ASP.NET Core Minimal API
  |     +--> Domain Service
  |     +--> Agent pipeline
  |     +--> SQLite
  |
  +--> React static assets
  |
  +--> Azure OpenAI / Semantic Kernel
  |
  +--> Azure Blob Storage
  |
  +--> Application Insights

The backend is ASP.NET Core 8 Minimal API, and the frontend is React + TypeScript. The local database is SQLite. Evidence Bundles are stored in Azure Blob Storage. For local development, I also added a local file fallback.

For Azure App Service, I zipped the backend publish output together with the React production build and deployed it. At first, QuickDeploy tried to run an Oryx server-side build and failed. It took me a while to notice that I needed to check the option that skips the server-side build.

How I Used Microsoft Technologies

Technology	How I used it
Azure OpenAI / Microsoft Foundry	Reasoning and structured JSON generation through role-specific AI steps
Semantic Kernel	Abstraction for Azure OpenAI chat completion calls
Azure App Service	Hosting ASP.NET Core API and React static files
Azure Blob Storage	Storing Evidence Bundle JSON / Markdown files
Application Insights	Checking App Service requests, failures, and latency
ASP.NET Core	Change Request API, state transitions, AuditEvent, and Agent pipeline control
React	UI for creating Change Requests, reviewing AI output, and downloading Evidence Bundles

Agent Pipeline

I avoided generating everything with one giant prompt. I split the roles and ran them in order.

Agent pipeline
  -> Change Intake
  -> DNS Risk Assessment
  -> Approval Brief
  -> Execution Plan
  -> Rollback Plan
  -> Evidence Report
  -> Evidence Bundle

Each line here is not a managed resource created in Azure AI Agent Service. It means a role-specific prompt execution unit inside ChangeProof Agent. In code, IAgentOrchestrator / AgentOrchestrator controls this agent pipeline. The UI only calls POST /api/change-requests/{id}/analyze once, but internally multiple PromptRuns are recorded.

Each AI output is handled as JSON, not just prose.

{
  "schemaVersion": "0.1.0",
  "agentRole": "dns_risk_assessor",
  "summary": "The DNS change may change the web destination.",
  "assumptions": [],
  "unknowns": [],
  "result": {
    "overallRiskLevel": "MEDIUM",
    "impactScope": [],
    "risks": [],
    "prechecks": []
  },
  "warnings": []
}

If JSON parsing fails, the app sends one repair prompt. If that still fails, it records the failure in PromptRun and AuditEvent. During development, I did see AGENT_PROMPT_FAILED. After reviewing the Azure OpenAI endpoint format and JSON mode settings, I eventually got AI generation working.

Evidence Bundle as the Center

In this MVP, the AI result is not just displayed on the screen and then forgotten. The app creates an Evidence Bundle for each Change Request.

It stores two main files.

• JSON: structured AI output and metadata
• Markdown: a human-readable Evidence Report

The blob names look like this.

{changeRequestId}/v{bundleVersion:0000}/{evidenceBundleId}.json
{changeRequestId}/v{bundleVersion:0000}/{evidenceBundleId}.md

This is the part that feels most meaningful as a product in this MVP. Instead of letting AI output disappear as a one-off chat response, the app bundles judgment, approval, procedure, and reporting into one artifact.

Of course, this does not complete audit compliance. There is no WORM storage, electronic signature, timestamping, permission management, or tamper detection yet. So the right wording is probably something like “it may be useful for audit preparation.”

Keeping Humans in the State Transitions

The state flow is:

DRAFT
  -> EXECUTION_READY
  -> APPROVED
  -> EXECUTED
  -> CLOSED

When Analyze succeeds, the request becomes EXECUTION_READY. After that, a human presses Mark as Approved, Mark as Executed, and Close.

I was quite conscious of this line. The AI does not approve things on its own, and it does not mark work as executed when no work has been done. It does not call a real DNS change API either.

AI prepares the material for judgment. Responsibility for approval and execution stays with the human.

Mapping to Hackathon Evaluation Criteria

Criterion	How ChangeProof Agent addresses it
Business impact	Makes small IT change work explainable for small IT companies and internal IT teams
Effectiveness as an AI agent	Splits risk assessment, approval text, execution steps, rollback, and evidence creation across an agent pipeline
Technical implementation	Uses Azure OpenAI, Semantic Kernel, ASP.NET Core, React, and Azure Blob Storage
Completeness	Implements demo UI, Evidence Bundle output, and Audit Events

Where I Got Stuck

Azure took more time than I expected.

First, some models are not available depending on region and project. Even if a model appears in the Foundry UI, that does not mean it can be deployed in your project. I had to check quota, region, model type, and whether it was available Direct from Azure.

With App Service, I also hit a case where my VM quota was 0 even though it was my first attempt. Free F1 was too weak for this setup, so I moved toward Basic B1.

For deployments, even though I thought I had uploaded the publish output, QuickDeploy ran an Oryx build, failed to detect a .csproj, and crashed. The fix was to check the “Skip Server-Side Build (Pre Built App)” option in the UI.

Finally, I also stumbled a bit on the Azure OpenAI endpoint. The Foundry UI sometimes shows an endpoint with /openai/v1, but Semantic Kernel’s Azure OpenAI connector expects the resource root format. I realized I should not paste the UI value as-is, so the app now normalizes everything after /openai.

https://<resource-name>.openai.azure.com/

Including these messy parts, it felt like a proper hackathon implementation that actually runs on Azure.

What Works Now

The current MVP can do the following:

• Create a DNS change request
• Generate a Risk Assessment with Azure OpenAI
• Generate an Approval Brief, Execution Plan, Rollback Plan, and Evidence Report
• Store Evidence Bundles as JSON / Markdown
• Store Audit Events
• Deploy to Azure App Service
• Store evidence in Azure Blob Storage
• Run the full demo flow in the React UI

What Is Still Missing

As a production service, it is still missing a lot.

• Authentication
• Permission management
• Multi-tenancy
• Evidence deletion and retention policies
• Key Vault / Managed Identity
• PromptRun search UI
• Read-only integration with Azure DNS / Cloudflare / Route53
• WORM Storage / Object Lock
• A more complete manual approval workflow

Put another way, the MVP was possible because I did not include these.

For this scope, I tried not to compromise on two lines: structure and store AI output, and keep human approval as an explicit state transition.

Next Steps

I built this as an MVP focused on DNS changes, but the same structure can be extended to other IT change work.

• Pre-release checks for websites
• Read-only integration with Cloudflare / Route53 / Azure DNS
• Change evidence for WordPress / PHP / SSL certificate updates
• Pre-review for Microsoft 365 / Entra ID setting changes
• Change management evidence reports for ISMS

Even if the use case changes, the core shape still applies: organize the change, identify what must be checked, and leave a record.

Conclusion

When people say AI agent, it is tempting to automate production operations all the way through. But I think the first practical point is a little earlier than that.

For small but painful-if-they-fail work like DNS changes, the operation itself is not always the main problem. Judgment, approval, rollback, and reporting tend to become vague. Having AI fill that gap feels fairly realistic to me.

ChangeProof Agent is still an MVP, but this direction felt promising.