Large language models (LLMs) have introduced an exciting new paradigm to the machine learning world in the past few years.

Research groups like AIS’s Advanced Research Concepts (ARC) team have been quick in exploring the range of possibilities with this new technology. However, aspiring AI developers and scientists often have no idea where to start learning about the science of large language models. In this blog post, we’ll review five seminal papers from the field and provide a brief reading guide so that you know which details to look for when you read them yourself.

CODEX: A Cluster-Based Method for Explainable Reinforcement Learning

AIS research scientists Timothy Mathes, PhD, and Andrés Colón have published a research paper titled CODEX: A Cluster-Based Method for Explainable Reinforcement Learning. This research was done in collaboration with Georgia Tech Research Institute and the Air Force Research Laboratory.

“Our research proposes a method for AI explainability that fuses techniques from Computer Vision and Natural Language Processing,” said Mathes. “We believe that investigating interdisciplinary approaches to understanding AI decision-making will be crucial for harnessing the power of intelligent systems.”

The team presented the paper during a virtual workshop at the International Joint Conference on Artificial Intelligence (IJCAI) Explainable AI Workshop on August 31, 2023.

Abstract:

Despite the impressive feats demonstrated by Reinforcement Learning (RL), these algorithms have seen little adoption in high-risk, real-world applications due to current difficulties in explaining RL agent actions and building user trust. We present Counterfactual Demonstrations for Explanation (CODEX), a method that incorporates semantic clustering, which can effectively summarize RL agent behavior in the state-action space. Experimentation on the MiniGrid and StarCraft II gaming environments reveals the semantic clusters retain temporal as well as entity information, which is reflected in the constructed summary of agent behavior. Furthermore, clustering the discrete+continuous game-state latent representations identifies the most crucial episodic events, demonstrating a relationship between the latent and semantic spaces. This work contributes to the growing body of work that strives to unlock the power of RL for widespread use by leveraging and extending techniques from Natural Language Processing.

In 2023, researchers at AIS proposed an internal research and development (IRaD) project seeking to develop novel technologies to deeply inspect Artificial Intelligence (AI) and Machine Learning (ML) based systems and algorithms. The team addresses this problem with a multi-disciplinary approach, applying knowledge from different internal groups at AIS including experts in AI/ML research and experts in the research and development of innovative software reverse engineering technologies. AIS’s AI/ML expertise spans the fields of Explainable AI (XAI), reinforcement learning, adversarial learning, safety and natural language processing.

“Before we can confidently rollout and leverage the power of AI/ML technologies, we need to better understand the algorithm’s process for achieving a certain conclusion,” said Georgia-Raye Moore, Advanced Research Program Manager. “This need has spawned into its own field within the industry and is referred to as ‘explainability’ or Explainable AI (XAI).”

AIS’s history of research in software analysis and inspection, which dates back to our founding, lends itself to an application in AI/ML based technologies.

“Decision makers need methods to deeply understand the risks and benefits of the AI/ML systems proposed for deployment within critical environments,” said Andrés Colón, Principal Investigator for the project. “The question our team seeks to address is: How can we better understand the underlying behaviors of often opaque AI/ML systems and build confidence prior to deployment?”

As part of this effort, AIS has been working to contribute to the space of XAI through the creation of a platform for unprecedented insight into AI/ML models.

“We are currently probing neural networks at the lowest levels during execution and surfacing this information to powerful analysis and visualization routines, clearing the path from opaque black-box neural network to unprecedented insight,” said Logan Boyd, Research Scientist and Technical Lead for the project. “The system is designed around extensibility so that researchers can build upon it and continue to push the state of the art in novel and interesting ways.”

The team is especially excited about this work as it lends to the testing and decomposition of complex AI/ML applications that are currently difficult to understand. Most importantly, this research supports making revolutionary AI/ML systems more trustworthy so they can confidently be deployed.

“It’s important for those adopting new AI/ML technologies to understand that models can have flaws, inherent limitations or blind spots,” said Gary Hamilton, Strategic Account Manager. “Our research aims to make these technologies safer by better understanding the system and enabling rapid adoption to ensure continued superiority.”

The team is eager to continue this research and the development of their tool in 2024. Learn more about advanced research at AIS: https://www.ainfosec.com/capabilities/advanced-research/