Agentic AI Readiness Checklist for Canadian Organizations

An organization is ready for agentic AI when it can answer six questions before the first agent is built: which task the agent owns, what data it may touch, which identity it runs as, which privacy law governs that data, how its behaviour will be tested, and what it will cost to run. If any one of those is unanswered, the pilot will usually stall at security review, not at the demo.

Agent demos are easy. The trouble starts when the demo is proposed for real data and real users. In our engagements, the pattern is that five decisions were deferred during the demo, and each one resurfaces as a blocker:

1. Nobody defined success, so there is no way to prove the agent is better than the current process.
2. The agent inherited broad data access from the developer’s test connection.
3. The agent runs as a shared or developer identity that the identity team never approved.
4. The privacy office was not consulted, and the PIA now has to be written backwards.
5. Nobody estimated running cost, so finance cannot approve the move to production.

None of these are model problems. They are governance decisions, and they are cheaper to make before the build. The checklist below is the one we use at the start of agent engagements to force those decisions early. Each of six dimensions is scored 0 to 2, and the total tells you whether to proceed to a production pilot, run a sandboxed pilot on synthetic data, or fix foundations first. The privacy section assumes Canadian law: FIPPA, PIPA or PIPEDA.

The readiness scorecard

Score each dimension honestly. A 2 means the decision is made and documented. A 1 means it is understood but not yet formalized. A 0 means nobody owns it.

Dimension	Score 0	Score 1	Score 2
Use case	“Try AI on something”	Task named, no baseline	Task, owner and baseline metric agreed
Data access	Agent sees whatever the connector sees	Sources listed, permissions unreviewed	Minimum sources, permissions cleaned, labels applied
Identity	Developer or shared credentials	Dedicated identity, broad roles	Own agent identity, sponsor named, resource-scoped roles
Privacy	No privacy review	Privacy office aware, no PIA	PIA done, data location and deployment type confirmed
Evaluation	Manual spot checks	Test set exists, no adversarial tests	Quality and safety evaluations, prompt injection tested, release threshold set
Cost	Unknown	Build cost known, run cost guessed	Run cost modelled per transaction, budget and alerts in place

The sections below explain what it takes to reach a 2 on each row. The scoring bands are at the end.

Is the use case worth automating?

We score candidate use cases on five factors:

• Frequency. A task done hundreds of times a month pays back faster than a quarterly one.
• Access. The fewer systems the agent needs, the shorter the security review.
• Measurability. Handling time, error rate or backlog size should be measurable today.
• Harm if wrong. Drafting a reply a human reviews is low harm. Changing a record in a system of record is not.
• Permitted data. If the task needs health or financial records, expect a longer privacy review.

A good first use case is a high-frequency task that touches few systems, has a measurable baseline, and produces output a human reviews before it takes effect. Internal knowledge and drafting agents usually qualify. Autonomous decisions on customer records usually do not.

Can the agent reach only the data it needs?

Data access is where oversharing surfaces. Agents that ground on SharePoint, Teams or file shares expose every permission mistake in those sources. Before the pilot, list the exact sources the agent needs, review their permissions, and apply sensitivity labels to what should stay out. We covered the same discipline for Microsoft 365 Copilot in rolling out Copilot with AI guardrails.

Does the agent have its own identity?

An agent needs its own identity, not a developer’s credentials or a shared service account. Microsoft Entra Agent ID gives agents a dedicated identity type in the directory, created from an agent identity blueprint and linked to a human sponsor accountable for its lifecycle (Microsoft Learn).

The design question that matters most is attended versus unattended. When an agent acts on behalf of a signed-in user through the on-behalf-of flow, it can only reach what that user is authorized for. When it runs unattended with client credentials, its access is bounded only by its own role assignments and application permissions (Foundry agent identity concepts). Unattended agents therefore need tighter scoping, resource-level roles, and access that expires. Without a user to limit them, their own roles are the only boundary.

We covered the full governance model, including sponsors, access packages and the identity change when a Foundry agent is published, in AI agent identity governance with Entra Agent ID. Confirm which of those features are available and licensed in your tenant before you design around them.

Is the processing allowed under FIPPA or PIPA?

Agents raise a privacy question that chat assistants often do not: they use personal information to take actions, sometimes for a purpose different from the one it was collected for. Canadian privacy law does allow agents to process personal information, within the same rules that apply to any system.

For BC public bodies, the privacy impact assessment under FIPPA is the gate. Public bodies can generally use personal information only for the purpose it was collected for or a consistent purpose, and if sensitive personal information will be stored outside Canada, a supplementary assessment is required. For private organizations, BC’s PIPA and the federal PIPEDA tie collection and use to purposes a reasonable person would consider appropriate, generally with consent, and keep the organization accountable for data it sends to service providers.

Two technical choices change the PIA:

• Model deployment type. In Microsoft Foundry, Global deployments may process prompts in any geography where the model is deployed. Standard (regional) deployments process them in the region of the resource (deployment types). If the PIA says Canada, the deployment has to be regional in Canada Central or Canada East. A Canadian resource region alone is not enough.
• Tooling location. Some evaluation and red teaming features run only in specific regions. Check the region list before sending real records through them, or use synthetic data.

Our post on Canadian data residency on Azure covers the FIPPA amendments and the Azure Policy controls that enforce residency, including a policy to deny Global deployments.

How will you know it works?

Nobody accountable can approve an agent that was never tested against a defined standard. Evaluation covers three things.

Quality. Does the agent complete the task correctly? Microsoft Foundry provides agent evaluators for intent resolution, task adherence and tool call accuracy, alongside general quality measures such as groundedness (agent evaluators). Build a test set of at least a few dozen real scenarios with expected outcomes, reviewed by the business owner.

Safety and prompt injection. Agents read documents and tool outputs, which makes them vulnerable to instructions hidden in that content. Testing means planting malicious instructions in what the agent reads and measuring how often it follows them. The Foundry AI Red Teaming Agent includes indirect prompt injection testing that plants attacks in mock tool outputs and measures an attack success rate (AI Red Teaming Agent). In production, Prompt Shields for documents screens grounding content for embedded attacks.

Action boundaries. Classify every action the agent can take. Read-only actions can run freely. High-risk actions, such as financial or medical decisions, need human confirmation. Irreversible actions, such as deletions, need disclosure and confirmation, or should not be available to the agent at all.

Set the release threshold before testing, not after. If the business agrees in advance on the pass rate the agent must meet, the go-live decision becomes a check, not a negotiation.

What will it cost to run?

Running cost is where agent programs surprise finance. It is the dimension finance asks about first and teams estimate last. An agent that calls a model several times per request, retrieves documents and invokes tools pays for all of it on every run.

Model the cost per completed task, not per month. Run the evaluation test set, measure token and tool consumption per scenario, and multiply by expected volume. Then check Microsoft’s current list prices for the model and deployment type, since the residency decision above can change the price.

For Foundry workloads, set an Azure Cost Management budget with alerts on the resource group that hosts the agent. For Copilot Studio agents, usage is measured in Copilot Credits, which administrators allocate to environments in the Power Platform admin center (Copilot Studio billing). Either way, the agent should have a named budget owner before it reaches production.

Reading your score

Add the six scores for a total out of 12:

• 10 to 12: proceed to a production pilot with real users and a limited scope.
• 6 to 9: run a sandboxed pilot on synthetic or de-identified data while the gaps are closed.
• Below 6: fix foundations first. Building now creates rework.

One override applies. A 0 on identity or privacy blocks production regardless of the total. An agent running on borrowed credentials, or processing personal information nobody has assessed, is not a pilot the identity team or the privacy office can sign off, however well it scores elsewhere.

Rescore after each gap is closed. The scorecard is most useful as a running record of which decisions are still open, not as a one-time grade.

Next steps

If an agent pilot is waiting on security or privacy review, the scorecard usually shows which decision is holding it up. Techrupt runs this readiness assessment, closes the gaps and takes the agent to production through our AI consulting practice in Vancouver.

To score your first use case with us, book a free consultation.