Aadim Nepal

Aadim Nepal

I am a senior undergraduate studying Mathematics and Computer Science at NYU Abu Dhabi, where I work with Professor Keith Ross on energy-based world models.

Previously, I worked on LLM reasoning and interpretability. Currently, I focus on how to get systems to learn from the physical world. I am interested in how principles from neuroscience and biology can inform the design of models that plan, predict, and generalize the way biological systems do.

Email / CV / Scholar / Twitter / GitHub / Blog / Miscellaneous

Recent News

Dec 2025 Layer Importance for Mathematical Reasoning and RL vs. Distillation accepted at the NeurIPS MATH-AI Workshop 2025.

Nov 2025 Presented Layer Importance for Mathematical Reasoning and Warm Up Before You Train at EMNLP 2025 in Suzhou, China.

Nov 2025 Warm Up Before You Train accepted at EMNLP 2025 main!

Oct 2025 Layer Importance for Mathematical Reasoning accepted at EMNLP BlackboxNLP 2025.

Jul 2025 Presented Multimodal Deep Learning for Stroke Prediction at EMBC 2025 in Copenhagen, Denmark.

Mar 2025 Multimodal Deep Learning for Stroke Prediction and Detection using Retinal Imaging accepted at EMBC 2025!

Jan 2025 Started my senior thesis with Prof. Keith Ross at the Deep Learning Lab, NYU Abu Dhabi, working on LLM interpretability and reasoning.

Jun 2023 Summer research at the Center for Cosmology and Particle Physics, NYU (New York), studying radio emissions of supermassive black holes using VLBA telescope data with Prof. Ingyin Zaw and Prof. Joseph Gelfand.

Research

I am interested in using model-based RL for System 2 planning. I study how systems can learn physical dynamics through self-supervised learning and world models. Much of my past work is in LLM reasoning and interpretability, but I am now focused on the foundational question of how to build human-level AI. Representative papers are highlighted.

	Layer Importance for Mathematical Reasoning is Forged in Pre-Training and Invariant after Post-Training Aadim Nepal, Safal Shrestha, Anubhav Shrestha, Minwu Kim, Jalal Naghiyev, Ravid Shwartz-Ziv, Keith Ross The 5th Workshop on Mathematical Reasoning and AI at NeurIPS, 2025 arXiv We find that a small number of layers are responsible for most of the mathematical reasoning ability in LLMs. These layers form during pre-training and do not change after post-training, regardless of the method used. Removing them drops math accuracy by up to 80%, while factual recall is spread more evenly across layers and holds up better.
	Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings Safal Shrestha, Minwu Kim, Aadim Nepal, Anubhav Shrestha, Keith Ross EMNLP, 2025 arXiv We show that training on simple logic puzzles before applying RLVR helps models learn to reason more effectively. This warm-up step improves performance even with limited data and compute, making it a practical approach for resource-constrained settings.
	Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning Minwu Kim, Anubhav Shrestha, Safal Shrestha, Aadim Nepal, Keith Ross The 5th Workshop on Mathematical Reasoning and AI at NeurIPS, 2025 arXiv We compare RLVR and distillation for improving LLM reasoning. RLVR improves accuracy on problems the model can already attempt but does not expand what it can solve. Distillation can improve both, but only when the teacher model brings in knowledge the student did not already have.
	Multimodal Deep Learning for Stroke Prediction and Detection using Retinal Imaging and Clinical Data Saeed Shurrab, Aadim Nepal, Terrence J. Lee-St. John, Nicola G. Ghazi, Bart Piechowski-Jozwiak, Farah Shamout EMBC, 2025 arXiv We build a multimodal model that combines retinal OCT scans, infrared fundus images, and clinical data to predict and detect stroke. Using all three modalities together improves AUROC by 5% over image-only models and by 8% over existing retinal foundation models.

Website template from Jon Barron.