A confident wrong answer sounds helpful. It is the worst possible outcome, because nobody thinks to check it. “I don’t know” sounds worse and is better: it can be defended, and it tells the user what to do next.
I gave this talk at the PowerUp! Meetup (Finland) on 2 September 2026, hosted by Microsoft in Keilaniemi, Espoo. The session was From Smart Answers to Trustworthy AI: Designing Governed Copilot Studio Agents, and this post is what was in it.
In enterprise use the challenge is not getting an agent to produce good answers. That part is easy. The challenge is making the answers predictable, sourced, current, and safe to rely on. That is not a property of the prompt. It is decided by what the agent is allowed to use in the first place.
Three ways a knowledge source fails you
More sources do not automatically mean better answers. In practice they mean three risks, and all three only show up in production.
- Uncontrolled sources. Duplicates, conflicting versions, accidental oversharing. The agent does not know which of two versions of a policy is in force, so it picks one and writes it up in a confident voice.
- Lifecycle blind spots. Stale, retired or deleted content. A document can be officially removed and the agent will still answer from it, because a copy sits in the index.
- Missing accountability. If a document has no owner, nobody has the authority to confirm that its content still holds. The answer cannot be defended even when it happens to be correct.
An agent can only be as trustworthy as the sources it is allowed to see.
A governed agent has an explicit contract
Once you start scoping the knowledge, scoping stops being a setting and becomes a design decision. The contract has four parts, and each one needs an answer before the agent ships.
Who. Which user or group is this content visible to?
What. Which approved content is even eligible?
Why. Who owns the decision that this content qualifies?
When. Is the content still valid?
The fourth one is the one that gets skipped. The first three are permission questions, and we are used to those. The fourth is a question about time, and it requires somebody to have thought ahead about what happens when a document expires.
Metadata turns a document into governed knowledge
In a real implementation, eligibility has to be machine-readable. It is not enough that a person knows the document is current, because that person is not there at the moment the agent retrieves it.
| Field | Value |
|---|---|
| Business process | Procurement |
| Country | Finland |
| Classification | Internal |
| Owner | Process team |
| Review date | 2026-10-01 |
| Status | Published |
The rule is simple: if the metadata does not prove eligibility, the content stays out. There is no default of “probably fine”, because that default is exactly what produces accidental oversharing.
The curated library loop: a tick box is the only door
I built the demo on Copilot Studio, SharePoint and Power Automate. The operating principle fits in one sentence: a person ticks a box, and that tick is the only route into the agent’s knowledge.
Documents live in an ordinary source library that the agent has no access to. When the responsible person ticks the Agent Knowledge field, Power Automate checks the metadata, the owner and the status, and only then does a copy appear in the curated library with the owner field filled in. The curated library is the agent’s only knowledge source. Untick the box and the document disappears from the agent as fast as it arrived.
Two settings take care of the rest. The agent’s useModelKnowledge and webBrowsing are both false, and they are set in YAML rather than in the portal. A portal toggle is easy to leave on and even easier to switch back on by accident. In YAML it is part of the source and it shows up in a diff.
The solution runs five flows. Three of them do nothing at curation time. They watch what is already inside: expiring content, orphaned files, and deleted sources. When one of them trips, the notification goes to a Teams channel rather than to an inbox, because the people who own this governance are already there and an email is one more thing to miss.
The reason a curated library is needed at all is not architectural elegance. It is what a shared Teams library actually looks like. A team gets a Teams site, everyone in the department drops documents into it, and the same library becomes the agent’s knowledge source because it was the obvious place to point at. Nobody sat down and asked whether the old versions are still in there. And you cannot fix that by telling people to be tidier: plenty of them do not distinguish a library from a folder in the first place, they just put the file in Teams.
How it got built
The five flows were written by Claude Code. My working loop is not the portal: I pull the agent out of Copilot Studio with the VS Code extension, Claude Code edits the YAML locally, and I push it back. That is also why useModelKnowledge and webBrowsing are set where they are. When the agent is a file on disk, an accidental setting change is a line in a diff instead of a click nobody remembers making.
The part that stayed manual was the connections. This was a new solution rather than an existing one, so there were no connection references to reuse, and somebody has to sit down and authorise each connector by hand. That is worth knowing before you plan the schedule: the building went fast, the wiring did not.
What the work actually feels like is a continuous conversation rather than delegation. It is much faster than doing it by hand. It is not hands-off.
Knowledge has a lifecycle, not just a location
Where a document sits tells you where it is. It does not tell you whether you may rely on it. Of the six lifecycle stages only one is eligible for the agent, and most of the governance work is making sure the other five stay out.
Stage five is the interesting one. When a document is retired it has to be excluded immediately, not at the next indexing run. A stale instruction is more dangerous than a missing one, because a missing instruction forces the user to ask a human and a stale one does not.
The orphaned file
This is the special case that appears the moment you follow the lifecycle all the way through: the file still exists, but the authority behind it does not.
The owner has left the organisation. The review date has passed. No fallback owner was assigned. The agent can still retrieve the file and will still answer from it, because nothing in the world it knows about has changed.
The governance decision has two halves, and the second half is what makes it usable. Remove the document from the agent’s approved scope, and route the user to the accountable process team. Removal on its own produces a dead end. Removal plus routing produces a next step.
The demo: three questions, three governed outcomes
The demo puts three questions to the agent, each landing in a different situation.
A valid, approved source. The agent answers and shows where the answer came from.
Expired knowledge. The agent refuses and explains the next step.
A missing owner. The agent excludes the source and routes the user to the process team.
Success is not the maximum number of answers. It is predictable behaviour. An agent that answers two out of three and is honest about why it will not answer the third is more production-ready than one that answers all three.
The claim is measured, not asserted
The demo data set has twenty documents, generated reproducibly by a script. It is deliberately messy rather than deliberately clean: six valid approved documents, four expired ones that still answer the same questions with full confidence but different figures, four drafts and personal notes, two mutually contradictory documents that both look current, and four pieces of pure noise such as a party invitation and a parking instruction. Seven of the twenty are curated.
When the evaluation ran, the agent found seven documents out of twenty. That number is exactly the number of curated documents. It is the whole thesis of the session expressed as one figure: scoping is not a setting you hope works, it is a property you can measure.
Two details about that data set are worth stealing. The generator is in version control, so the set is recreated bit for bit before every rehearsal and the score on stage is the score you practised with. And the subject matter is dragons and unicorns, so that nobody in the audience can mistake the demo for real customer data.
The twenty documents and the sixty evaluation questions were both generated by Claude Code on my instructions, and reviewed by me before they counted for anything.
One thing the copy does not bring with it
A question from the room landed on the part of this design that is easiest to get wrong. Copilot Studio respects SharePoint permissions when a library is the knowledge source, which is exactly what you want. But the curated library is a second library, and a copy that lands there does not inherit the access rules of the library it came from.
So the curated copy has to be given the same permissions as its source, deliberately, as part of the flow. Get that wrong and you have built a governance mechanism whose only real effect is to widen access. That is worse than not building it, because it looks careful.
This gets harder, not easier, when the curated library serves a whole organisation and documents arrive into it from several source libraries with different audiences.
The last 15 percent: prove the behaviour
Building a governed agent is design work. Keeping it governed is measurement work. What came out of that is a way of working I call Test-Driven Agent Development. The name is mine: I went looking for prior art under that term and found very little, which is either a good sign or a warning.
The loop has four steps. The product owner defines the real questions the agent should handle. Evaluations run against the agent. Claude Code implements a targeted change. Then a control run repeats the same evaluation with nothing changed.
The fourth step is the one without which the first three mislead you.
Same agent. Same questions. Same grader. No changes in between. The total went from 24 to 31, which is a seven point improvement produced by doing nothing.
Without the control run somebody would have drawn a conclusion from that. If the previous change had landed just before round two, it would have looked like a successful fix. In the other direction the same noise looks like a regression, and then you go and fix something that was never broken.
The order is: repeat first, diagnose second, change third.
When an evaluation goes red, classify before you fix. Four classes covered practically everything I ran into, and each has a different repair.
An agent defect is fixed in the agent. A measurement defect is fixed in the test. A structural gap means the agent is being asked for something it was never built to do, and that is a product design decision. A content gap means the authoritative information does not exist anywhere, and you do not fix that in the agent. You fix it by writing the missing document.
The tests decide which change comes next. The human decides which failure is worth acting on.
Four principles to apply tomorrow
Scope before prompts. Define exactly what the agent may know. This is the first decision, not the last bit of polish.
Metadata before retrieval. Eligibility has to be provable and machine-readable.
Lifecycle before launch. Design for expiry, deletion and owner loss before the agent reaches production.
Evidence before change. Repeat the evaluations and classify the failures before you fix anything.
In closing
A successful enterprise agent is not the one that answers the most questions. It is the one that answers only the questions it is allowed to answer.
The slides are freely available: From Smart Answers to Trustworthy AI.