Singing with Text-To-Music Conversion

Introduction to Text-to-Music Conversion in AI-DOL

The AI-DOL project introduces an innovative leap in virtual performances with its text-to-music conversion technology. This advanced feature enables AI-DOLs to transform written lyrics and melodies into fully realised songs, sung in the virtual idols’ distinctive voices. Through this technology, AI-DOL can create original music in response to fan suggestions, themes, or directly from creative text inputs, making the music creation process interactive and fan-inclusive.

Objectives of Text-to-Music Conversion in AI-DOL

The primary goals behind incorporating text-to-music conversion are:

• Creative Fan Engagement: To allow fans to play a role in the music creation process, submitting lyrics or themes that AI-DOL can turn into songs.

• Diverse Musical Output: To produce a wide array of music styles and genres, reflecting the diverse preferences of the fan base and showcasing the versatility of AI-DOLs.

Proof of Concept

A recent showcase featured AI-DOL performing a song cover requested by a fan, utilising voice cloning technology to demonstrate our innovative approach to fan engagement and the versatility of our virtual idols.

A sneak peek video of AI-DOL's debut song which is generated by AI.

How Text-to-Music Conversion Works in AI-DOL

Text-to-music conversion utilises a combination of NLP for understanding and structuring the text inputs and sophisticated music generation algorithms to compose melodies that match the lyrical content. AI-DOLs then perform these compositions using vocal synthesis technology, creating songs that resonate with fans’ inputs and preferences.

A flowchart depicting the text-to-music conversion process, highlighting AI’s role in interpreting text, composing music, and performing the final song within the AI-DOL ecosystem.

Applications of Text-to-Music Conversion in AI-DOL

• Fan-Driven Song Creation: Fans can contribute lyrics, which AI-DOLs convert into complete musical pieces, blurring the lines between fan and artist in the creative process.

• Themed Music Challenges: AI-DOLs participate in themed challenges, where fans submit text related to specific themes or emotions, and AI-DOLs create songs based on these prompts.

Future Directions

Moving forward, AI-DOL aims to push the boundaries even further. Our future direction focuses on developing Text-To-Music technology that enables AI-DOL to craft unique songs based on fan requests.