Statistical Learning in Structured Markov Chains and MDPs

Junghyun Lee
Oct 27, 2020 project

Project #1. Clustered State Space

Nearly Optimal Latent State Decoding in Block MDPs

  • Accepted to AISTATS 2023
  • Joint work with Se-Young Yun (KAIST AI) and Yassir Jedra, Alexandre Proutière (KTH EECS).
  • Some empirical analyses:
  • Accepted to KSC 2023. Joint work with Se-Young Yun (KAIST AI).
    • Extended version accepted to KIISE Transactions on Computing Practices (invited). Also joint work with Se-Young Yun (KAIST AI).

Project #2. Mixture of Markov Chains and MDPs

Near-Optimal Clustering in Mixture of Markov Chains

  • arXiv preprint arXiv:2506.01324
  • Joint work with Yassir Jedra (MIT EECS -> ICL EEE), Alexandre Proutière (KTH EECS), and Se-Young Yun (KAIST AI).
RL MDP Statistics Probability Theory Markov Chain Community Detection Clustering
Junghyun Lee
Junghyun Lee
PhD Student

PhD student at GSAI, KAIST, jointly advised by Profs. Se-Young Yun and Chulhee Yun. Research focuses on interactive machine learning, particularly at the intersection of RLHF and preference learning, and statistical analyses of large networks, with an emphasis on community detection. Broadly interested in mathematical and theoretical AI and related problems in mathematics.

Posts

Two preprints now available on arXiv!
Two preprints on two very different topics now available :) Near-Optimal Clustering in Mixture of Markov Chains Joint work with Yassir Jedra (MIT EECS -> ICL EEE), Alexandre Proutière (KTH EECS), and Se-Young Yun (KAIST AI) AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners Again led by amazing undergrad intern, Woosung Koh (Yonsei Univ.
Junghyun Lee
Jun 2, 2025 1 min read
Project Project
Two papers accepted to CKAIA 2023!
Two papers (Fair Streaming Principal Component Analysis: Statistical and Algorithmic Viewpoint, Communication-Efficient Collaborative Heterogeneous Bandits in Networks) is accepted to CKAIA 2023! These are joint works with Hanseul Cho, Chulhee Yun, Laura Schmid, and Se-Young Yun (KAIST AI).
Junghyun Lee
Jul 13, 2023 1 min read
Project
Best oral presentation for KSC 2022 + invitation to submit to KTCP
Our KSC 2022 paper (Preliminary Empirical Analyses of Clustering in Block MDPs) has been selected as one of the best oral presentataions! Additionally, we have been invited to submit an extended version of our conference paper to the KIISE Transactions on Computing Practices (KTCP)!
Junghyun Lee
Feb 8, 2023 1 min read
Project
One invited paper and one poster accepted for presentation at the INFORMS APS 2023!
Our recent AISTATS paper, Nearly Optimal Latent State Decoding in Block MDPs, has been invited to be presented at the invited session “Stochastic learning in structured systems” at INFORMS APS 2023 (chair: Prof.
Junghyun Lee
Jan 20, 2023 1 min read
Project
One paper accepted to AISTATS 2023!
One paper (Nearly Optimal Latent State Decoding in Block MDPs) is accepted to AISTATS 2023! This is joint work with my advisor Se-Young Yun (KAIST AI), and Yassir Jedra and Alexandre Proutiére (KTH EECS).
Junghyun Lee
Jan 20, 2023 1 min read
Project
I've successfully defended my MSc thesis!
I’ve successfully defended my MSc thesis, titled Near-Optimal Clustering in Block Markov Decision Processes! Special thanks to the thesis committee: Se-Young Yun, Chulhee Yun, and Kee-Eung Kim (KAIST AI). Thesis as well as presentation file coming soon!
Junghyun Lee
Dec 15, 2022 1 min read
Project
One paper accepted to KSC 2022!
One paper (Preliminary Empirical Analyses of Clustering in Block MDPs) is accepted to KSC 2022! This is joint work with Se-Young Yun (KAIST AI). More details to come!
Junghyun Lee
Nov 21, 2022 1 min read
Project
Our preprint on theoretical analysis of block MDP now available on arXiv!
Our preprint Nearly Optimal Latent State Decoding in Block MDPs is now available on arXiv! (https://arxiv.org/pdf/2208.08480.pdf) This is joint work with Yassir Jedra and Alexandre Proutiere (KTH EECS), and Se-Young Yun (KAIST AI).
Junghyun Lee
Aug 19, 2022 1 min read
Project
One paper accepted to CKAIA 2022 + selected as one of the best papers!
One paper (Near-Optimal Clustering in Block MDPs with Implications on Reward-Free RL) is accepted to CKAIA 2022! This is joint work with Yassir Jedra and Alexandre Proutiere (KTH DCS), and Se-Young Yun (KAIST AI).
Junghyun Lee
Jul 28, 2022 1 min read
Project

Publications

On the Estimation of Linear Softmax Parametrized Markov Chains
tbd
Kunwoo Na, Junghyun Lee, Se-Young Yun
Cite Project Slides
On the Estimation of Linear Softmax Parametrized Markov Chains
Empirical Analyses of Corruption in the Clustering of Block MDPs
We show that a simple trick of randomly corrupting the trajectories in Block MDPs allow for us to use the the clustering algorithm proposed of Jedra et al. (2023) for general classes of MDPs.
Junghyun Lee, Se-Young Yun
PDF Cite Project
Empirical Analyses of Corruption in the Clustering of Block MDPs
Nearly Optimal Latent State Decoding in Block MDPs
First theoretical analysis of model estimation and reward-free RL of block MDP, without resorting to function approximation frameworks. Lower bounds and algorithms with near-optimal upper bound are provided.
Yassir Jedra, Junghyun Lee, Alexandre Proutière, Se-Young Yun
PDF Cite Code Project Poster Slides
Nearly Optimal Latent State Decoding in Block MDPs
Preliminary Empirical Analyses of Clustering in Block MDPs
We empirically validate the clustering algorithm proposed in (Jedra et al., 2022).
Junghyun Lee, Se-Young Yun
PDF Cite Project Slides
Preliminary Empirical Analyses of Clustering in Block MDPs