LLM

Introduction to Reinforcement Learning with Human Feedback (RLHF): A Theoretically Biased Overview
Event Weekly OptiML Lab Group Meeting Short summary In this talk, I will first (somewhat rigorously) introduce the framework of reinforcement learning with human feedback (RLHF). Then I will go over three recent breakthroughs in the analysis and improvement of RLHF.
Nov 30, 2023
Regularized Online RLHF with Generalized Bilinear Preferences
We consider the problem of contextual online RLHF with general preferences, where the goal is to identify the Nash Equilibrium. We …
Junghyun Lee, Minju Hong, Kwang-Sung Jun, Chulhee Yun, Se-Young Yun
Cite
Regularized Online RLHF with Generalized Bilinear Preferences
Learning to Reason in LLMs by Expectation Maximization
Large language models (LLMs) solve reasoning problems by first generating a rationale and then answering. We formalize reasoning as a …
Junghyun Lee, Branislav Kveton, Anup Rao, Subhojyoti Mukherjee, Ryan A. Rossi, Sunav Choudhary, Alexa Siu
PDF Cite
Learning to Reason in LLMs by Expectation Maximization
AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners
Self-Taught Reasoners (STaR), synonymously known as Rejection sampling Fine-Tuning (RFT), is an integral part of the training pipeline …
Woosung Koh, Wonbeen Oh, Jaein Jang, MinHyung Lee, Hyeongjin Kim, Ah Yeon Kim, Joonkee Kim, Junghyun Lee, Taehyeon Kim, Se-Young Yun
PDF Cite Code
AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners