Speech Modeling with a Hierarchical Transformer Dynamical VAE

Xiaoyu LinXiaoyu BieSimon LeglaiveLaurent GirinXavier Alameda-Pineda

Mar 2023

摘要

The dynamical variational autoencoders (DVAEs) are a family oflatent-variable deep generative models that extends the VAE to model a sequenceof observed data and a corresponding sequence of latent vectors. In almost allthe DVAEs of the literature, the temporal dependencies within each sequence andacross the two sequences are modeled with recurrent neural networks. In thispaper, we propose to model speech signals with the Hierarchical TransformerDVAE (HiT-DVAE), which is a DVAE with two levels of latent variable(sequence-wise and frame-wise) and in which the temporal dependencies areimplemented with the Transformer architecture. We show that HiT-DVAEoutperforms several other DVAEs for speech spectrogram modeling, while enablinga simpler training procedure, revealing its high potential for downstreamlow-level speech processing tasks such as speech enhancement.

展开全部

图表提取