Generative Artificial Intelligence (GenAI) and its applications in third-party cyber risk management programs

Many security experts and leaders are worried about artificial intelligence risks. The emergence of this new technology brings new tools to the field, but managing risks, including AI risks themselves, with a technology that we are still trying to understand, is not always the best idea. One reason why governance, risk, and compliance (GRC) experts often stick to tried-and-true approaches is that it's difficult to reliably discern patterns in information gathered by frequently changing methods.

There are two main questions. One is how to effectively employ AI in risk management and, more specifically for us here at Tenchi Security, in third-party cyber risk management (TPCRM) programs. The other is how to manage the risks stemming from the use of GenAI tools by third parties.

This post will tackle the first question, but we will come back to this topic in a future post to talk about GenAI use by third parties, too.

It would be pretentious on our part to say that we know what GenAI will be good for or where its use is "allowed" and where it isn't. That is likely something we will collectively learn through educated guesses and much trial and error. Some will be rewarded for their endeavor, while others will be left with nothing but painful lessons, and it's not always clear who is being bold and who is being reckless until the dust settles.

To complicate things further, GenAI is a moving target. It's not the same technology that it was a year ago, and it's likely not going to be the same a year from now. If those changes are not accounted for, GenAI may generate contradictory assessments for similar or identical risk profiles when consistency would typically be expected.

If it's not possible to tell whether results have changed because of updates to the underlying software or because of an actual change in the environment, that is a problem.

But technology exists to solve problems, so this is not about ruling out AI. It's about understanding its strengths and leveraging them where it makes the most sense for each business.

Where and how GenAI can excel

Current GenAI models are not truly "intelligent" and cannot think, but what sets them apart from most other software is the capability to handle data in more unstructured and natural states. In simpler terms, AI can interact with data meant for people, not computers.

Creating structured data for computer systems requires a set of standards and formats, but it's not possible to foresee every situation. Thus, people use tools like writing, drawings, and photos, which can convey information to other people but would be meaningless to computers unless someone were to manually convert them to a predefined format or add metadata that a system can use.

Using AI, it's possible to rearrange many human-readable sets of data, even if they do not follow a specific structure. GenAI can "look" at photos to find objects to answer if something exists in the image, or to filter pictures that match a given query even without any preexisting metadata. GenAI can attempt to interpret drawings and turn them into text, or read documents and repackage the information they contain to feed traditional systems and databases.

Of course, GenAI can also make it easier for people to quickly extract insights from large volumes of data. This is why many have found success in adopting GenAI for web searches or research: since scientific papers are written for humans and not computers, keyword-based searches weren't as effective as GenAI can be.

While generative AI and "vibe coding" have made many headlines, they're also extensions of this behavior. It is by rearranging data and naturally communicated knowledge that GenAI transforms the way we work.

Unfortunately, AI is probabilistic and will frequently "hallucinate" by adding unrelated and undesirable data, leading to inaccuracies. It's worth pointing out that even the term "hallucination" is misleading, anthropomorphizing models which are not actually perceiving and reasoning. But we are forced to use it since it's become the de facto standard for describing generative AI mistakes.

Furthermore, the computing power required by the more advanced GenAI models effectively binds them to the cloud and third-party platforms.

The road to adopting GenAI and managing its risks requires tackling these two main issues: accuracy and confidentiality. Both are sensitive topics in risk management.

AI, compliance, and risk management

As we pointed out in our post about GRC Engineering, many governance and risk management approaches focus on due diligence and formalities while shunning automation and technology. Because AI can function in scenarios that rely on human-readable data, it seems like a perfect fit to bridge this gap. 

While GenAI has made many GRC professionals realize that automation is a possibility, the best path to accomplish this does not always involve AI. With the right approach, many GRC and TPCRM tasks can be automated without GenAI or very deliberate implementations of simpler GenAI models, avoiding its critical pitfalls for this line of work and reducing costs.

Whenever possible, designing new processes geared for automation from the ground up should be the preferred option. For example, using GenAI to automate writing responses to self-assessment questionnaires (SAQs) makes more sense when the question is about something that cannot be inferred from information that an autonomous system could retrieve from the vendor's IT ecosystem.

As per the Gartner® Predicts 2026: Third-Party Cybersecurity Risk Management Evolves for the AI Era research, "The problem with using GenAI to aid both the respondent and requester is not merely the increased risk of hallucinations. GenAI-driven analysis of GenAI-driven inputs leads to output degradation, error amplification, and eventually model collapse."

Despite that, there is a trend of growth in the adoption of AI in this scenario. Gartner also predicts that “By 2028, 70% of organizations and vendors will use GenAI for both completing responses to TPCRM questionnaires and analyzing completed questionnaires, rendering the outputs increasingly unusable and disconnected from actual risk indicators.”

Instead, Gartner suggests that “Use GenAI and AI techniques to redirect human resources to more valuable risk management activities. Do not confuse the improved automation of check-box activities (e.g., records that must be kept for the sake of compliance) with improved risk management.”

For us, this means evaluating if AI can be used in places where deep data analysis is required to such an extent that the work was too expensive or time-consuming to be performed by humans in the first place. This goes beyond automating existing processes.

After that is done, there will still be room to explore how AI can fill the remaining gaps. Measuring the expected accuracy and determining how to make the best use of human supervision are very important steps here.

If businesses simply implement GenAI into the processes designed to be performed by people, the results may be underwhelming. GenAI not only does not solve the reliability issues in existing processes, but it will also add its own set of inaccuracies. While traditional automation usually reduces inaccuracies, this is often a byproduct of datasets being standardized and prepared the system to consume and, therefore, is not a given with AI.

This required standardization means that automation often requires more work in the early stages. It’s tempting to say that GenAI allows us to skip that extra preparatory work, we have to keep in mind that what’s happening is more like delaying the work to the end of the task, when human review and supervision happens. Both approaches have their place, but it's necessary to understand where the costs will show up.

‍

A responsible approach to GenAI in third-party risk management

There are real and serious reasons that GRC tasks haven't been automated. Risk management and compliance usually involve high-level concepts, unique circumstances, and tailor-made policies crafted to accommodate the specific set of requirements that a business must follow. When it comes to regulations, there are many special cases and exceptions.

This means that not everything can be described in a simple and deterministic format, and that there is a legitimate use for GenAI in third-party cyber risk management.

Tenchi Security is adding GenAI into its platform in a way that respects the accuracy demanded by risk management efforts. We inform users about the limitations of the tool in the context where this information is relevant, allowing them to decide how to interpret the data processed by the AI. Plus, we only do thorough testing comparing the accuracy of the GenAI output by comparing it to human expert output on a representative sample of inputs.

Currently, our platform uses GenAI to review answers and any attached documents sent by vendors in response to self-assessment questions or information requests. A large-language model (LLM) analyzes the answers to identify the specific parts of (potentially large) answers and attachments that are relevant in the context of the question, and any potential gaps not covered by the provided answer. This empowers the human reviewers, saves them precious time, and allows them to more confidently conduct follow-ups with third-parties as needed

We will add more AI-powered features as we discover more scenarios where it reliably brings value. The core of our continuous, inside-out monitoring solution is fully automated and highly accurate. 

The most important aspect of our vision is being able to utilize this high-quality ground truth level data that is unique to Zanshin's inside-out scanning to ensure the models can produce optimal outputs. As the legendary Peter Norvig put it:  "More data beats clever algorithms, but better data beats more data."

Balancing that with less reliable data in a way that combines accuracy with coverage is where the real challenge lies. We are committed to building advanced and efficient approaches to TPCRM that are aligned with business needs and the best technology for each use case.

‍