Our research

Driven by the need for transparency and trust in AI, the Reasoning with Machines AI Lab is advancing the benchmarking and evaluation of large language models to unlock their potential across various domains. Our approach combines human-computer interaction (HCI) with mechanistic interpretability. By both studying how humans engage with AI systems and exploring the inner mechanics of these models, we aim to develop a deep, nuanced understanding of their functioning and limitations. This dual focus allows us to

• Evaluate the reasoning capabilities of AI models, ensuring that they align with human expectations and can perform tasks effectively
• Investigate the practical applications of AI in domains like healthcare, where trust and transparency are paramount, and human-computer interaction, where user experience and model reliability are essential.

Our current research areas include:

• Benchmarking AI reasoning abilities. Current benchmarks often fail to capture the complexities of real-world reasoning. We design new evaluation methods and datasets to measure AI performance across logical, mathematical and common sense reasoning tasks.
• Understanding failure modes in LLMs. Despite their impressive capabilities, LLMs struggle with consistency, logical coherence and robustness in reasoning-heavy tasks. We analyse these failure points and develop strategies to mitigate them.
• Human-AI reasoning interactions. AI models increasingly assist humans in decision-making, but their reasoning processes are often opaque. We study how humans interpret and rely on AI-generated reasoning, aiming to make these interactions more transparent and trustworthy.
• Dynamic evaluation frameworks. As AI systems evolve, traditional benchmarks quickly become obsolete. We explore new methodologies, such as dynamic benchmarking and adversarial testing, to 2 of 3 ensure AI reasoning is assessed in a rigorous and meaningful way.

We cultivate a collaborative research environment that encourages innovation and learning for postgraduate students. Through peer learning, code sharing, and hands-on training in data science and STEM fields, we accelerate their learning, helping them to acquire critical skills and engage with cutting-edge research in a dynamic, team-oriented setting.