I am a first-year PhD student in Manning College of Information & Computer Sciences, University of Massachusetts Amherst, asvised by Prof. Hamed Zamani and Prof. Wenting Zhao. Before that, I was a research assistant at Generative AI Research Lab (GAIR) to explore Generative AI, fortunately working with Prof. Pengfei Liu. I received the M.S. degree in Computer Technology at Institute of Computing Technology (ICT) of Chinese Academy of Sciences (CAS) supervised by Prof. Jiafeng Guo in 2024 and the B.E. degree in computer science and technology from Shanghai Maritime University in 2021.
Research Interests: My primary research interests include natural language processing, large language models, and machine learning. Specifically, My current research focuses:I am happy to collaborate and/or answer questions about my research. If you are interested in research collaboration or have any inquiries about my experience, please send me an email.
Scientific reasoning is critical for developing AI scientists and supporting human researchers in advancing the frontiers of natural science discovery. However, the open-source community has primarily focused on mathematics and coding while neglecting the scientific domain, largely due to the absence of open, large-scale, high-quality, verifiable scientific reasoning datasets. To bridge this gap, we first present TextbookReasoning, an open dataset featuring truthful reference answers extracted from 12k university-level scientific textbooks, comprising 650k reasoning questions spanning 7 scientific disciplines. We further introduce MegaScience, a large-scale mixture of high-quality open-source datasets totaling 1.25 million instances, developed through systematic ablation studies that evaluate various data selection methodologies to identify the optimal subset for each publicly available scientific dataset. Meanwhile, we build a comprehensive evaluation system covering diverse subjects and question types across 15 benchmarks, incorporating comprehensive answer extraction strategies to ensure accurate evaluation metrics. Our experiments demonstrate that our datasets achieve superior performance and training efficiency with more concise response lengths compared to existing open-source scientific datasets. Furthermore, we train Llama3.1, Qwen2.5, and Qwen3 series base models on MegaScience, which significantly outperform the corresponding official instruct models in average performance. In addition, MegaScience exhibits greater effectiveness for larger and stronger models, suggesting a scaling benefit for scientific tuning. We release our data curation pipeline, evaluation system, datasets, and seven trained models to the community to advance scientific reasoning research. |
@article{fan2025megascience, title={MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning}, author={Fan, Run-Ze and Wang, Zengzhi and Liu, Pengfei}, year={2025}, journal={arXiv preprint arXiv:2507.16812}, url={https://arxiv.org/abs/2507.16812} }
Amid the expanding use of pre-training data, the phenomenon of benchmark dataset leakage has become increasingly prominent, exacerbated by opaque training processes and the often undisclosed inclusion of supervised data in contemporary Large Language Models (LLMs). This issue skews benchmark effectiveness and fosters potentially unfair comparisons, impeding the field's healthy development. To address this, we introduce a detection pipeline utilizing Perplexity and N-gram accuracy, two simple and scalable metrics that gauge a model's prediction precision on benchmark, to identify potential data leakages. By analyzing 31 LLMs under the context of mathematical reasoning, we reveal substantial instances of training even test set misuse, resulting in potentially unfair comparisons. These findings prompt us to offer several recommendations regarding model documentation, benchmark setup, and future evaluations. Notably, we propose the "Benchmark Transparency Card" to encourage clear documentation of benchmark utilization, promoting transparency and healthy developments of LLMs. we have made our leaderboard, pipeline implementation, and model predictions publicly available, fostering future research. |
@article{xu2024benchmarking, title={Benchmarking Benchmark Leakage in Large Language Models}, author={Xu, Ruijie and Wang, Zengzhi and Fan, Run-Ze and Liu, Pengfei}, year={2024}, journal={arXiv preprint arXiv:2404.18824}, url={https://arxiv.org/abs/2404.18824} }
The quality of finetuning data is crucial for aligning large language models (LLMs) with human values. Current methods to improve data quality are either labor-intensive or prone to factual errors caused by LLM hallucinations. This paper explores elevating the quality of existing instruction data to better align with human values, introducing a simple and effective approach named ReAlign, which reformats the responses of instruction data into a format that better aligns with pre-established criteria and the collated evidence. This approach minimizes human annotation, hallucination, and the difficulty in scaling, remaining orthogonal to existing alignment techniques. Experimentally, ReAlign significantly boosts the general alignment ability, math reasoning, factuality, and readability of the LLMs. Encouragingly, without introducing any additional data or advanced training techniques, and merely by reformatting the response, LLaMA-2-13B's mathematical reasoning ability on GSM8K can be improved from 46.77% to 56.63% in accuracy. Additionally, a mere 5% of ReAlign data yields a 67% boost in general alignment ability measured by the Alpaca dataset. This work highlights the need for further research into the science and mechanistic interpretability of LLMs. We have made the associated code and data publicly accessible to support future studies at https://github.com/GAIR-NLP/ReAlign. |
@article{fan2024reformatted, title={Reformatted Alignment}, author={Fan, Run-Ze and Li, Xuefeng and Zou, Haoyang and Li, Junlong and He, Shwai and Chern, Ethan and Hu, Jiewen and Liu, Pengfei}, year={2024}, journal={arXiv preprint arXiv:2402.12219}, url={https://arxiv.org/abs/2402.12219} }
University of Massachusetts Amherst, 2025.09 - Present
Manning College of Information & Computer Sciences
Ph.D. in Computer Science, supervised by Prof. Hamed Zamani and Prof. Wenting Zhao.
Shanghai Jiao Tong University, 2023.05 - 2025.06
Generative AI Research Lab (GAIR)
Research assistant, supervised by Prof. Pengfei Liu.
JD Explore Academy, 2022.12 - 2023.05
Research intern, supervised by Dr. Liang Ding.
University of Chinese Academy of Sciences, 2021.09 - 2024.06
Institute of Computing Technology
M.S. in Computer Science and Technology, supervised by Prof. Jiafeng Guo.
Shanghai Maritime University, 2017.09 - 2021.06
B.E. in Computer Science and Technology
2024: Excellent Master’s Graduation Thesis, Institute of Computing Technology
2021: Excellent Bachelor's Graduation Thesis, Shanghai Maritime University
2021: Excellent Graduate, Shanghai Maritime University
2019, 2020, 2021: First Class Scholarship, Shanghai Maritime University