Suyash
Maniyar

Measuring Reward Hacking and Reasoning-Answer Decoupling Under Confounded Optimization

Abhishek Mishra*, Armaan Sandhu*, Suyash Maniyar*

• arXiv
• Paper PDF
• Poster
• Course report (IPPO)
• Code

• Post-trained 10 LLMs (Qwen2.5, Llama 3.x) with GRPO on a confounded reward, inducing reward hacking.
• Identified a backdoor-like failure that arises from fully correct training data and that endpoint metrics cannot detect, then designed an evaluation protocol with reasoning-answer decoupling and out-of-domain probes to catch it.
• Red-teamed checkpoints: the hacked model chose the harmful option 84% of the time in the rewarded position, against 4% for controls and 0% once positions were swapped.

AI-Generated Lecture Slides for Improving Slide Element Detection and Slide Image Retrieval

• Paper
• Project website

• Built an LLM-guided synthetic data pipeline (GPT-4, 18 layouts) that generates 4,400 auto-labelled slides across 16 element classes with no manual labelling.
• Fine-tuned CLIP on synthetic data to R@1 43 on real slide retrieval, beating 31k real out-of-domain slides (R@1 31), and lifted LayoutLMv3 detection from 27.3 to 33.9 mAP.

Enhancing Legal LLMs with Metadata-Enriched Retrieval and DPO

• arXiv
• Project report

• Built hybrid FAISS retrieval fusing dense and BM25 scores, cutting retrieval mismatch from 65% to 10%.
• Engineered metadata-enriched chunking from case summaries and details, improving recall@10 by 23%.
• Aligned Llama 3.2 1B with DPO on 3.8K preference pairs, cutting over-refusal from 53.2% to 1.5%.

Is the Refusal Direction Mostly Harmfulness?

• Reproduced the claim that LLMs encode harmfulness and refusal separately on 8 models (Llama-3, Qwen2.5, Gemma-3, 3B to 27B) using nnsight activation interventions.
• Showed the separation fails: after removing the refusal component, the harm direction keeps 76% of its causal effect.
• Broke the proposed latent safety guard with a nanoGCG suffix, dropping recall to 40-74%.

AI Alignment as a Continual Learning Problem

• Studied whether LLMs can both learn and unlearn behaviors with LoRA, training 6 sequential adapters on paired safety, bias, truthfulness and privacy tasks (PKU-SafeRLHF, BBQ, TruthfulQA, TOFU).
• Found all 4 merge strategies collapsed refusal to 0.005, then built a zero-parameter centroid router that beat the best merge by up to 52 points and held MMLU at 0.406 against 0.170.

Agentic RL Self-Play for Biomedical QA

Advised by Prof. Hong Yu

• Designed an ontology-driven agentic RL self-play framework with dual question and answer LLM agents, using MCP for tool-grounded verification, improving biomedical QA by 12% over fine-tuning.

Data-Driven Haptic Modelling 1st place

• Poster

• Devised a method for force modelling of inhomogeneous objects in haptics, reaching 95% accuracy.
• Presented "Barycentric Interpolation for Force Modelling on Visco-Elastic Inhomogeneous Objects" and won first place at the Youth Conclave 2022.

Advanced Data Science Associate

ZS Associates

• Developed a model that predicts which patients will switch from ongoing drug treatments, using clinical data and an ensemble of ML (Bayes, XGBoost) and DL (TabNet) models, improving F1 by 11%.
• Engineered an end-to-end MLOps pipeline on AWS SageMaker, orchestrated with Airflow and Docker, automating drift detection and retraining across 5+ models and cutting manual deployment effort by 60%.
• Ran SHAP analysis that helped optimize marketing, with projected savings of up to $500K.
• Built patient cohorts from 100M+ claims records by applying business rules in SQL on Amazon Redshift.
• Quantified the impact of advertisements with A/B testing and Double ML, informing campaign spend.

ML Engineer

Decimal Point Analytics

• Built an XGBoost supply chain model for a leading FMCG client, forecasting SKU-level demand from lag and rolling-window features engineered in Spark, cutting monthly inventory waste by 15%.
• Designed an anomaly detection pipeline that predicts defects in industrial machinery from time-series data.
• Shipped a RAG chatbot for Q&A with LangChain and Pinecone, adding knowledge graphs for a 9% accuracy gain and FAQ response caching that cut inference cost by 30%.
• Automated internal web scraping by fine-tuning Llama 2 (7B/13B) with LoRA for text extraction and using GPT-4 for code generation, reducing downtime from 12 hours to 3 minutes.
• Built a computer vision car damage segmentation system, improving claim processing efficiency by 75%.
• Compressed few-shot object detection models by 70% with knowledge distillation for edge deployment.

Machine Learning Intern

RaapidAI

• Built a table detection and structure recognition pipeline for medical document images, fine-tuning CascadeTabNet, DETR and DiT to 76% mAP, with augmentation adding 7% on a private gold-standard set.

AI/ML Intern

Decimal Point Analytics

• Built an end-to-end accident detection and damage segmentation pipeline for CCTV footage.
• Compared Vision Transformer, YOLO and other architectures, reaching an F1 score of 87%.

EmbedServe: embedding serving for recommender systems

• Built a high-performance embedding serving system in C++ with RocksDB and dynamic request batching.
• Reduced memory by 3.5× and p99 latency by 3.5× at about 2,000 req/s through tiered storage and INT8 quantization.
• Reached 45% of peak GPU bandwidth with a fused Triton kernel, matching TorchRec within 20% p99 latency while using 1.55× less memory.

Slide Scribe Winner

• Devpost

• Real-time voice Q&A over live lecture slides for visually impaired students. Detects slide changes through CLIP embedding similarity and answers with Gemini, on a Flutter client and FastAPI backend.

UniTrade: campus marketplace

• GitHub

• Built a campus-only marketplace on a Next.js/TypeScript client and a Node/Express REST API.
• Implemented live bidding over WebSockets (Socket.IO), pushing bid updates to connected clients in real time.
• Added natural-language search that parses text queries into structured category, price and condition filters.

Cohort analysis and engagement segmentation

• Built an engagement classification framework on Edpuzzle watch-time and quiz analytics for the Intro to Engineering course, using k-means to flag the least engaged 10% of students for instructor follow-up.
• Compared quiz performance between cohorts with and without video assignments.

MS in Computer Science

University of Massachusetts Amherst, CGPA 4.0/4.0

Coursework: Advanced NLP, Software Engineering, Systems for Deep Learning, Neural Networks for Neuroscience, Advanced Machine Learning, Systems for Data Science, AI Alignment

B.Tech in Electrical Engineering

Indian Institute of Technology, Jodhpur, CGPA 8.2/10.0

Coursework: Calculus, Linear Algebra, Data Structures & Algorithms, Machine Learning, Deep Learning, DLOps, Natural Language Understanding, Computer Networks, Intro to Cognitive Science, Mobile and Pervasive Computing, Biosensors