Japanese pitch conversion for voice morphing based on differential modeling

Abstract In this paper, we convert the pitch contours predicted by a TTSsystem that models a source speaker to resemble the pitch con-toursofatargetspeaker. Whenthespeakingstylesofthespeak-ers are very different, complex conversions such as adding ordeletingpitchpeaksmayberequired. Ourmethoddoesthecon-versions by modeling the direct pitch features and differentialpitch features at the same time based on linguistic features. Thedifferential pitch features are calculated from matched pairs ofsource and target pitch values. We show experimental results inwhich the target speaker’s characteristics are successfully mod-eledbasedonaverylimitedtrainingcorpus. TheproposedpitchconversionmethodstretchesthepossibilitiesofTTScustomiza-tion for various speaking styles. Index Terms : pitch conversion, voice conversion, voice mor-phing, speech synthesis, differential modeling. 1. Introduction Voice conversion changes the characteristics of the voice of anSPS (SPeaker Source) to those of an SPT (SPeaker Target) forvarious applications. One important application is to build cus-tomized text-to-speech (TTS) systems for different companies,soaTTSsystemwitheachcompany’sfavoritevoicecanbecre-ated quickly and inexpensively by modifying the speech corpusof some original speaker.Spectra and prosody are the two major characteristics ofvoice. For spectral conversion, recent work such as [1, 2]has achieved significant improvements in the naturalness andsimilarity of the voices converted using only a limited amountof training data. However, not much research has been doneinto prosody conversion. Most spectral conversion researchuses simple linear transformations for the prosody. It is truethat the detailed prosody difference is sometimes difficult forlisteners to distinguish [3], especially when the speakers aremonotonously reading for TTS corpus recording. However, togenerate TTS voices with a lively colloquial speaking style, re-production of the detailed prosody characteristics is important.Our objective is to reproduce the SPT’s speaking style ofpitch contours based on limited training data. We assume 100sentences as training data is a reasonable amount to require forthe SPT’s speech corpus. A speech corpus with that size caneasily be recorded in a thirty-minute recording session. We donot assume the existence of a parallel corpus, which can be adifficult condition to satisfy. We focus on the pitch changesaround the syllable level, because the important pitch changesin Japanese are mainly at the syllable level.Figure 1 illustrates examples of pitch contour pairs that theproposed pitch conversion method can handle. They are (a)asymmetrical slope changes, (b) adding or deleting peaks, and(c) adding or deleting phrase-final rises.