Who Is Accountable for AI Decisions in Insurance? Carriers and Vendors Weigh In.

When an insurer uses an AI model to price, underwrite, or settle a claim, who is accountable for the decision? At ITC Vegas 2026, leaders from across the insurance and technology ecosystem came back to a consistent answer: the carrier remains accountable, while the vendor shares responsibility for how the model is explained, governed, and used.

The discussion brought together Stephanie Kuczynski, Director of Risk Analytics at ZestyAI; Sandy Ulrich, SVP of Information Technology at Pharmacists Mutual; Rajat Sharma, Chief Revenue Officer at InsureMO; and Kobi Bendelak, CEO of Insurtech Israel, with Daniel Wolfe, Editor-in-Chief of Digital Insurance, moderating.

As AI becomes more deeply embedded in underwriting, pricing, and claims, that distinction matters more. Many of the models carriers rely on come from outside vendors, raising important questions around ownership, oversight, and accountability. Who is ultimately responsible for the decision? What does the vendor owe the carrier? And what does a regulator expect when a model influences an outcome?

Who is accountable when an insurer uses a vendor's AI model?

The carrier remains ultimately accountable for the decision, regardless of whether the model was developed internally or provided by a third party. But that does not absolve the vendor of responsibility. Carriers need enough transparency, traceability, and explainability to understand how the model works and defend its use.

ZestyAI's Stephanie Kuczynski framed that responsibility around three requirements: traceability, so a carrier can show how a score was produced; accountability, so ownership of the outcome is clear; and explainability, so the decision can be understood by a regulator or other external stakeholder. Those considerations have to be addressed before a model goes live, not after a decision is challenged.

Pharmacists Mutual's Sandy Ulrich drew an important distinction between running the technology and owning the business outcome. IT may support or operate the model, but the underwriting or claims leader acting on its output is still responsible for the decision that follows.

Rajat Sharma of InsureMO agreed that the carrier remains the final point of accountability, while noting that the maturity of governance still varies significantly across the market. Larger carriers are generally further along in formalizing these processes, while many smaller organizations are still building them.

Kobi Bendelak of Insurtech Israel added the vendor perspective. Early-stage technology companies may not have the same governance infrastructure as an insurer, which can shape what carriers reasonably require during vendor evaluation. It does not, however, change where ultimate accountability sits.

What does "human in the loop" actually mean for insurance AI?

Human oversight only works if the person reviewing an AI recommendation has the authority and expertise to challenge it. A human in the loop should own the outcome, not simply approve the output.

Pharmacists Mutual's Sandy Ulrich made that distinction explicit. The person reviewing an AI-driven recommendation needs enough context and judgment to question it when necessary. That requires training, not just a review step. Pharmacists Mutual has used intern programs and internal IT champions to help raise that baseline across the organization. Ulrich summed it up simply: train people to be thinkers.

Stephanie Kuczynski argued that vendors share responsibility for making that oversight possible. A model provider cannot simply deliver a score and step away. The carrier needs to understand how to interpret the result, trace how it was produced, and explain it when a regulator or other stakeholder asks why a decision was made. If the carrier cannot explain the model, the vendor has not finished the job.

Rajat Sharma of InsureMO added that the level of human oversight should depend on the maturity of the use case. In more established applications of AI, a human reviewer may add less value. In areas such as underwriting, where both the technology and the surrounding processes are still evolving, human judgment remains more important.

Kobi Bendelak offered a different view, arguing that the pace of AI development may eventually make traditional human review impractical and that AI agents could take on more of that oversight role.

How do regulators approach AI in insurance, and how does it differ by state?

For insurers, AI oversight is still highly state-specific. The questions regulators ask can vary by jurisdiction, but the focus is increasingly consistent: how does the technology affect the policyholder, and can the carrier explain the decision?

Stephanie Kuczynski spoke from ZestyAI's experience working with state regulators on behalf of carrier customers. There is no single U.S. rulebook for AI in property insurance, and each department of insurance brings its own expectations. In some states, regulators may focus on the age or source of the data behind a property score. In others, scrutiny may center on how the model affects rates, eligibility, or underwriting decisions.

Across those differences, Kuczynski said the conversation increasingly comes back to the consumer. Regulators want to understand how a model influenced an individual outcome, which makes traceability and explainability critical. A carrier should be able to trace a decision back to the relevant inputs and explain the result in plain language.

That becomes especially important when a model contributes to an adverse action. If a policyholder is non-renewed, surcharged, or declined, the carrier needs an appeal process that gives a qualified person the ability to review the decision and change the outcome when appropriate.

Regulatory readiness also has to be considered before deployment. Insurers need to know whether a model can be used in the states where they write business and whether that status remains current as requirements evolve.

This is where regulatory experience becomes a meaningful part of vendor evaluation. ZestyAI's models hold more than 500 regulatory approvals across state departments of insurance, supported by direct engagement with regulators in the markets where carrier customers operate. That infrastructure helps carriers respond more quickly and confidently when questions arise about how a property-level score was produced or used.

Kobi Bendelak added that the challenge is not limited to the U.S. In Europe, AI regulation can be even more complex, and startups may not fully confront those requirements until late in the sales or implementation process.

How should insurers manage data before deploying AI?

AI accountability starts with the data behind the model. If the inputs are inconsistent, incomplete, or difficult to trace, every downstream governance question becomes harder to answer.

Pharmacists Mutual's Sandy Ulrich emphasized the need for a single source of truth. Before deploying AI, insurers need confidence in the accuracy of their data and a clear understanding of which decisions depend on it. That becomes especially important as carriers experiment with large language models and other tools that may draw from multiple internal systems.

Rajat Sharma of InsureMO extended that point to the broader data environment. It is not enough for data to be clean for transactions; it also has to support regulatory scrutiny. That means building a data strategy that preserves the information needed to reconstruct how a decision was made, rather than relying on institutional memory after the fact.

Stephanie Kuczynski connected that directly to explainability. Traceability starts with the inputs: the carrier should be able to identify which imagery date, parcel boundary, model version, or other underlying data produced a particular score. Without that record, explaining the outcome to a regulator becomes much more difficult.

Ulrich also drew an important distinction between internal and customer-facing uses of AI. Tools used for internal enablement may warrant one level of governance, while applications that directly affect customers require a higher standard. The same technology can carry very different risks depending on where and how it is used.

What changes in AI accountability over the next 6 to 12 months?

The near-term shift is likely to be less about whether insurers use AI and more about how rigorously they evaluate, govern, and operationalize it.

Stephanie Kuczynski expects vendor vetting to become more demanding across the market. Large carriers already tend to require model validation, documentation, and regulatory review as part of procurement. Over the next year, more mid-size and smaller carriers are likely to adopt similar expectations as regulators ask tougher questions about how AI is selected and used.

For vendors, that raises the bar. The ability to explain how a model works, document its inputs and outputs, support regulatory review, and demonstrate that it can be governed in practice will increasingly become part of the buying decision.

Sandy Ulrich expects the most immediate gains from AI to come through efficiency, with teams first using the technology to streamline workflows before moving toward more complex decision support as confidence grows.

Rajat Sharma pointed to a similar evolution in process design. AI is already taking on a larger share of tasks such as software development, but he expects the bigger change to come when organizations begin redesigning workflows around AI rather than simply inserting it into existing processes.

Kobi Bendelak took a longer-term view, predicting that increasingly autonomous AI systems will become a much bigger part of the industry conversation over the next year.

One takeaway from each panelist

The panel closed with one final takeaway from each speaker, and together they captured the main themes of the discussion:

• Stephanie Kuczynski, ZestyAI: Accountability is shared, but the carrier remains the final point of responsibility. Vendors need to be transparent about how their models work and support carriers in explaining and governing their use.
• Sandy Ulrich, Pharmacists Mutual: Human oversight only works when people are equipped to challenge the technology. As she put it: train people to be thinkers.
• Rajat Sharma, InsureMO: Strong AI governance starts with strong data. Every other control depends on having accurate, traceable information underneath the model.
• Kobi Bendelak, Insurtech Israel: As AI systems become more autonomous, the industry will need to rethink what oversight and accountability look like.

For carriers, the practical implication is clear: a vendor's accountability should be demonstrated, not assumed. ZestyAI's models hold more than 500 regulatory approvals, and more than half of the top 100 carriers use them. Both depend on the ability to show how a property-level score was produced, explain how it should be used, and support carriers when regulators ask questions.

See how carriers deploy ZestyAI models.

FAQ: AI accountability in insurance

Who is accountable when an insurer uses a third-party AI model?
The carrier remains ultimately accountable for the underwriting, pricing, claims, or other business decision made using the model. The vendor also has important responsibilities, including providing the transparency, traceability, documentation, and support the carrier needs to understand and govern how the model is used.

What does AI explainability mean in insurance?
Explainability is the ability to communicate, in understandable terms, how a model contributed to a decision. That includes understanding the relevant inputs, how the model uses them, and why a particular outcome followed. Traceability complements explainability by preserving a record of the data, model version, and other information behind an individual result.

What should insurers require from an AI vendor?
Insurers should look for more than technical performance. A strong AI vendor should be able to document how its models work, provide traceability into individual results, support regulatory review, establish clear data governance and continuity practices, and train carrier teams to interpret and explain the technology appropriately.

What does “human in the loop” mean in insurance AI?
Human oversight is meaningful only when the person reviewing an AI-driven recommendation has the knowledge and authority to challenge it. The appropriate level of oversight will depend on the use case, but simply adding a human approval step does not by itself create effective governance.

How does regulation of AI in insurance differ by state?
Insurance regulation remains highly state-specific, and departments of insurance may focus on different aspects of an AI model or its use. Insurers need to understand whether a model can be used in the states where they operate and be prepared to explain how it affects individual policyholders. Working with a vendor that has established regulatory experience can make that process significantly easier to manage.

What is the foundation of AI accountability in insurance?
Reliable, traceable data. Insurers need confidence in the accuracy of the information feeding a model and the ability to reconstruct which inputs, data sources, and model versions contributed to a decision. Without that foundation, explainability and governance become much harder.

See how insurers are putting ZestyAI to work.