Gartner Survey Revealed 34% of Organizations Are Already Using or Implementing AI Application Security Tools

Mismanagement Of AI Risks by Organizations Can Lead to Their Models Not Performing as Intended, And In The Worst Case, Cause Human Or Property Damage

Thirty-four percent of organizations are either already using or implementing artificial intelligence (AI) application security tools to mitigate the accompanying risks of generative AI (GenAI), according to a new survey from Gartner, Inc. Over half (56%) of respondents said they are also exploring such solutions.

The Gartner Peer Community survey was conducted from April 1 to April 7 among 150 IT and information security leaders at organizations where GenAI or foundational models are in use, in plans for use, or being explored.

Twenty-six percent of survey respondents said they are currently implementing or using privacy-enhancing technologies (PETs), ModelOps (25%) or model monitoring (24%) (see Figure 1).

Figure 1. Organizations Using or Planning to Use Tools to Address Risks Related to Generative AI (Percentage of Respondents)

Source: Gartner Peer Community (September 2023)

“IT and security and risk management leaders must, in addition to implementing security tools, consider supporting an enterprise-wide strategy for AI TRiSM (trust, risk and security management),” said Avivah Litan, Distinguished VP Analyst at Gartner. “AI TRiSM manages data and process flows between users and companies who host generative AI foundation models, and must be a continuous effort, not a one-off exercise to continuously protect an organization.”

IT Is Ultimately Responsible for GenAI Security

While 93% of IT and security leaders surveyed said they are at least somewhat involved in their organization’s GenAI security and risk management efforts, only 24% said they own this responsibility.

Among the respondents that do not own the responsibility for GenAI security and/or risk management, 44% reported that the ultimate responsibility for GenAI security rested with IT. For 20% of respondents, their organization’s governance, risk, and compliance departments owned the responsibility.

Top-of-Mind Risks

The risks associated with GenAI are significant, continuous and will constantly evolve. Survey respondents indicated that undesirable outputs and insecure code are among their top-of-mind risks when using GenAI:

• 57% of respondents are concerned about leaked secrets in AI-generated code.
• 58% of respondents are concerned about incorrect or biased outputs.

“Organizations that don’t manage AI risk will witness their models not performing as intended and, in the worst case, can cause human or property damage,” said Litan. “This will result in security failures, financial and reputational loss, and harm to individuals from incorrect, manipulated, unethical or biased outcomes. AI malperformance can also cause organizations to make poor business decisions.”

Gartner clients can read more in the report “Generative AI Security and Risk Management.”

About Gartner IT Symposium/Xpo

Gartner IT Symposium/Xpo is the world’s most important gathering for CIOs and other IT executives. IT executives rely on these conferences to learn how to unleash the possibility of the technology, insights and trends shaping the future of IT and business. Follow news, photos and video coming from Gartner IT Symposium/Xpo on the Gartner Newsroom, on X using #GartnerSYM, Instagram and LinkedIn.

Upcoming dates and locations for Gartner IT Symposium/Xpo include:
October 16-19 | Orlando, FL
November 6-9 | Barcelona, Spain
November 13-15 | Tokyo, Japan
November 28-30 | Kochi, India

About Gartner for Cybersecurity Leaders
Gartner for Cybersecurity Leaders equips security leaders with the tools to help reframe roles, align security strategy to business objectives and build programs to balance protection with the needs of the organization. Additional information is available at https://www.gartner.com/en/cybersecurity.

Follow news and updates from Gartner for Cybersecurity Leaders on X and LinkedIn using #GartnerSEC. Visit the Gartner Newsroom for more information and insights.

About Gartner

Gartner, Inc. (NYSE: IT) delivers actionable, objective insight that drives smarter decisions and stronger performance on an organization’s mission-critical priorities. To learn more, visit gartner.com