File size: 1,623 Bytes
4c1253a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | ---
library_name: transformers
pipeline_tag: text-generation
license: mit
---
# GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models
This repository contains the model checkpoint for **GDSD** (Guided Denoiser Self-Distillation), as introduced in the paper [GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models](https://huggingface.co/papers/2605.29398).
GDSD is a reinforcement learning (RL) framework designed to improve the denoiser of diffusion large language models (dLLMs). It reduces RL to a likelihood-free self-distillation objective by matching the dLLM's denoiser logits to an advantage-guided self-teacher. This approach bypasses the training–inference mismatch (TIM) biases common in ELBO-based methods and leads to more stable training dynamics.
## Resources
- **Paper:** [GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models](https://arxiv.org/abs/2605.29398)
- **GitHub Repository:** [GaryBall/GDSD](https://github.com/GaryBall/GDSD)
## Citation
If you find GDSD helpful, please consider citing the following work:
```bibtex
@misc{tang2026gdsdreinforcementlearningguided,
title={GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models},
author={Xiaohang Tang and Keyue Jiang and Che Liu and Qifang Zhao and Xiaoxiao Xu and Sangwoong Yoon and Ilija Bogunovic},
year={2026},
eprint={2605.29398},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2605.29398},
}
``` |