References

Models and training

• A. Vaswani et al. Attention Is All You Need. NeurIPS 2017. https://arxiv.org/abs/1706.03762
• A. Radford et al. Language Models are Unsupervised Multitask Learners. OpenAI, 2019.
• T. Brown et al. Language Models are Few-Shot Learners. NeurIPS 2020. https://arxiv.org/abs/2005.14165
• B. Zhang, R. Sennrich. Root Mean Square Layer Normalization. NeurIPS 2019. https://arxiv.org/abs/1910.07467
• R. Xiong et al. On Layer Normalization in the Transformer Architecture. ICML 2020. https://arxiv.org/abs/2002.04745
• D. Hendrycks, K. Gimpel. Gaussian Error Linear Units (GELUs). 2016. https://arxiv.org/abs/1606.08415
• I. Loshchilov, F. Hutter. Decoupled Weight Decay Regularization. ICLR 2019. https://arxiv.org/abs/1711.05101
• I. Loshchilov, F. Hutter. SGDR: Stochastic Gradient Descent with Warm Restarts. ICLR 2017. https://arxiv.org/abs/1608.03983
• R. Pascanu, T. Mikolov, Y. Bengio. On the Difficulty of Training Recurrent Neural Networks. ICML 2013. https://arxiv.org/abs/1211.5063
• L. Xue et al. ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models. TACL 2022. https://arxiv.org/abs/2105.13626

Audio generation

• A. Agostinelli et al. MusicLM: Generating Music From Text. 2023. https://arxiv.org/abs/2301.11325
• Z. Borsos et al. AudioLM: a Language Modeling Approach to Audio Generation. IEEE/ACM TASLP 2023. https://arxiv.org/abs/2209.03143
• P. Dhariwal et al. Jukebox: A Generative Model for Music. 2020. https://arxiv.org/abs/2005.00341
• J. Liu et al. DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism. AAAI 2022. https://arxiv.org/abs/2105.02446
• A. Défossez et al. High Fidelity Neural Audio Compression (EnCodec). TMLR 2023. https://arxiv.org/abs/2210.13438
• R. Kumar et al. High-Fidelity Audio Compression with Improved RVQGAN (DAC). NeurIPS 2023. https://arxiv.org/abs/2306.06546
• J. Copet et al. Simple and Controllable Music Generation (MusicGen). NeurIPS 2023. https://arxiv.org/abs/2306.05284
• N. Zeghidour et al. SoundStream: An End-to-End Neural Audio Codec. IEEE/ACM TASLP 2021. https://arxiv.org/abs/2107.03312
• J. Ho, T. Salimans. Classifier-Free Diffusion Guidance. 2022. https://arxiv.org/abs/2207.12598
• A. Fan, M. Lewis, Y. Dauphin. Hierarchical Neural Story Generation. ACL 2018. https://arxiv.org/abs/1805.04833
• R. Pope et al. Efficiently Scaling Transformer Inference. MLSys 2023. https://arxiv.org/abs/2211.05102
• J. Kong, J. Kim, J. Bae. HiFi-GAN. NeurIPS 2020. https://arxiv.org/abs/2010.05646
• H. Siuzdak. Vocos: Closing the Gap between Time-Domain and Fourier-Based Neural Vocoders. 2023. https://arxiv.org/abs/2306.00814

Data

• D. Stoller, S. Durand, S. Ewert. End-to-End Lyrics Alignment for Polyphonic Music Using an Audio-to-Character Recognition Model. ICASSP 2019. https://arxiv.org/abs/1902.06797
• S. Durand, D. Stoller, S. Ewert. Similarity-based Audio-Lyrics Alignment of Multiple Languages. ICASSP 2023. https://arxiv.org/abs/2306.07744
• I. Manco et al. The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation. NeurIPS 2023 Workshop. https://arxiv.org/abs/2311.10057
• M. Defferrard, K. Benzi, P. Vandergheynst, X. Bresson. FMA: A Dataset for Music Analysis. ISMIR 2017. https://arxiv.org/abs/1612.01840

Signal processing

• D. Griffin, J. Lim. Signal Estimation from Modified Short-Time Fourier Transform. IEEE TASSP 1984. https://doi.org/10.1109/TASSP.1984.1164317
• N. Perraudin, P. Balazs, P. Søndergaard. A Fast Griffin-Lim Algorithm. WASPAA 2013. https://doi.org/10.1109/WASPAA.2013.6701851
• J. O. Smith. Digital Audio Resampling Home Page. CCRMA, Stanford. https://ccrma.stanford.edu/~jos/resample/
• J. O. Smith. Spectral Audio Signal Processing. CCRMA, Stanford. https://ccrma.stanford.edu/~jos/sasp/
• M. Slaney. Auditory Toolbox, Version 2. Interval Research TR 1998-010.
• S. Lloyd. Least Squares Quantization in PCM. IEEE Trans. Information Theory 1982. https://doi.org/10.1109/TIT.1982.1056489
• A. Oppenheim, R. Schafer. Discrete-Time Signal Processing. 3rd ed., Pearson, 2010.

GPU programming

• V. Volkov, J. Demmel. Benchmarking GPUs to Tune Dense Linear Algebra. SC 2008. https://doi.org/10.1109/SC.2008.5214359
• T. Dao et al. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. NeurIPS 2022. https://arxiv.org/abs/2205.14135
• NVIDIA. cuda-oxide. https://github.com/NVlabs/cuda-oxide
• D. Kirk, W. Hwu, I. El Hajj. Programming Massively Parallel Processors. 4th ed., Morgan Kaufmann, 2022.
• NVIDIA. CUDA C++ Programming Guide. https://docs.nvidia.com/cuda/cuda-c-programming-guide/
• G. Steele, D. Lea, C. Flood. Fast Splittable Pseudorandom Number Generators. OOPSLA 2014. https://doi.org/10.1145/2714064.2660195