Sample Complexity of Learning in Structured Markov Chains and MDPs

Junghyun Lee
Oct 27, 2020 project

Project #2. TBD

Work in progress. Joint work with Yassir Jedra (MIT EECS), Alexandre Proutière (KTH EECS), and Se-Young Yun (KAIST AI).

Project #1. Nearly Optimal Latent State Decoding in Block MDPs

Accepted to AISTATS 2023. Joint work with Se-Young Yun (KAIST AI) and Yassir Jedra, Alexandre Proutière (KTH EECS).

Some empirical analyses:

  • Accepted to KSC 2023. Joint work with Se-Young Yun (KAIST AI).
    • Extended version accepted to KIISE Transactions on Computing Practices (invited). Also joint work with Se-Young Yun (KAIST AI).
RL MDP Statistics Probability Theory Markov Chain
Junghyun Lee
Junghyun Lee
PhD Student

PhD student at GSAI, KAIST, jointly advised by Prof. Se-Young Yun and Prof. Chulhee Yun. Interested in mathematical and theoretical AI, i.e., essentially any machine learning challenges necessitating mathematical analysis. Recently focused on statistical problems arising from RLHF, including interactive machine learning and low-dimensional structure recovery.

Posts

Two papers accepted to CKAIA 2023!
(to be filled out…)
Junghyun Lee
Jul 13, 2023 1 min read
Project
Best oral presentation for KSC 2022 + invitation to submit to KTCP
(to be filled out…)
Junghyun Lee
Feb 8, 2023 1 min read
Project
One invited paper and one poster accepted for presentation at the INFORMS APS 2023!
(to be filled out…)
Junghyun Lee
Jan 20, 2023 1 min read
Project
One paper accepted to AISTATS 2023!
(to be filled out…)
Junghyun Lee
Jan 20, 2023 1 min read
Project
I've successfully defended my MSc thesis!
(to be filled out…)
Junghyun Lee
Dec 15, 2022 1 min read
Project
One paper accepted to KSC 2022!
(to be filled out…)
Junghyun Lee
Nov 21, 2022 1 min read
Project
Our preprint on theoretical analysis of block MDP now available on arXiv!
(to be filled out…)
Junghyun Lee
Aug 19, 2022 1 min read
Project
One paper accepted to CKAIA 2022 + selected as one of the best papers!
(to be filled out…)
Junghyun Lee
Jul 28, 2022 1 min read
Project

Publications

On the Estimation of Linear Softmax Parametrized Markov Chains
tbd
Kunwoo Na, Junghyun Lee, Se-Young Yun
Cite Project Slides
On the Estimation of Linear Softmax Parametrized Markov Chains
Empirical Analyses of Corruption in the Clustering of Block MDPs
We show that a simple trick of randomly corrupting the trajectories in Block MDPs allow for us to use the the clustering algorithm proposed of Jedra et al. (2023) for general classes of MDPs.
Junghyun Lee, Se-Young Yun
PDF Cite Project
Empirical Analyses of Corruption in the Clustering of Block MDPs
Nearly Optimal Latent State Decoding in Block MDPs
First theoretical analysis of model estimation and reward-free RL of block MDP, without resorting to function approximation frameworks. Lower bounds and algorithms with near-optimal upper bound are provided.
Yassir Jedra, Junghyun Lee, Alexandre Proutière, Se-Young Yun
PDF Cite Code Project Poster Slides
Nearly Optimal Latent State Decoding in Block MDPs
Preliminary Empirical Analyses of Clustering in Block MDPs
We empirically validate the clustering algorithm proposed in (Jedra et al., 2022).
Junghyun Lee, Se-Young Yun
PDF Cite Project Slides
Preliminary Empirical Analyses of Clustering in Block MDPs