Multi-language Adaption(in common)
Last updated
Last updated
After completing the editing on the "Assets Upload" page, click "Next" to enter the "Multi-language Adaption" editing page. The left side displays the translation results for all materials, while the right side shows the preview window and the status of language and size variations.
When the produced video needs to be used in different countries or regions, the Multi-language Adaption feature allows for quick video translation, enabling the simultaneous batch export of multiple language versions of the video.
π‘π‘π‘Multi-language Adaption Recommended Workflowπ‘π‘π‘
1οΈβ£ Confirm Translation Content Ensure the translated text is accurate.
2οΈβ£ Review Original Subtitles Check if the recognized subtitles in the original language need fine-tuning and whether speaker assignments are correct. β If adjustments are made, remember to update the translation.
3οΈβ£ Select Voice Tone for Multi-language Different tones generate different dubbing speeds. β Update the dubbing.
4οΈβ£ Fine-Tune Translations Adjust the translated content based on the dubbing speed and context.
5οΈβ£ Export After confirming all details are correct, proceed with the export.
Check the desired languages and content to be translated in the popup window. The available content includes:
Translatable content included in the layout template, such as button text, product names, promotional messages, etc.
Translatable content from the main video, including video subtitles and voice-overs, as well as text replacements and translations generated by AI removal.
After confirming the content, click "Translate" and wait for a moment.
π Supported Dub Recognition Methods: Recognize Script from VoiceoverγRecognize Script from SubtitlesγUpload File.
Recognize Script from VoiceoverοΌThe default recognition method. Smart recognition based on the audio in the video.
Recognize Script from SubtitlesοΌRecognition based on subtitles in the video.
Please note: By default, subtitles will be detected and removed within a fixed range (the default position is described in the pop-up window).
If the position does not match, you can customize it using "Add" (similar to the "Video AI Removal" feature).
If the range has already been marked during the AI Removal step, no need to mark it again here (the detection range matches the AI subtitle removal range.).
Upload FileοΌRecognition based on an uploaded subtitle file.
File format requirements: SRT or ASS.
Please note: The uploaded subtitle file duration must be less than the video duration; otherwise, the excess part will be truncated!
Once the translation content is generated, you can still Update Subtitles to re-recognize the subtitles.
π‘ You can also modify the language or content that needs translation (add or delete) at the indicated position.
In the "Layout Templates" translation section, you can translate and adjust the text (such as button text, product names, promotional messages, etc.) included in the layout template.
Click on the corresponding language's text box (the preview window on the right will also switch accordingly), and manually fine-tune the translation results.
Click the AI button to use AI tools for text processing, including retranslation, content enrichment, content trimming.
It also supports entering custom commands in the text box, such as shortening the text to within 10 characters.
After AI rewriting, you can choose to replace, retranslate, discard the translation, or switch between different translation results.
Click the symbol button to modify the text style; it supports selecting different fonts based on the language.
π‘ This text style will be applied to all languages synchronously.
In the "Video Content" translation section, you can adjust the translation of the content included in the added main video.
The "Video Content" is divided into one or multiple translation sections, depending on the number of videos selected for variation. For each added main video, there will be a corresponding translation section for that video.
Each translation section for the main video consists of 4 modules:
Text Replacement: If an AI text replacement task was submitted for the video on the "Assets Upload" page, this module will show the replaced text and its corresponding translation.
Text Translation: If an AI text translation task was submitted for the video on the "Assets Upload" page, this module will display the original text and its corresponding translation.
Subtitle Translation: If an AI subtitle translation task was submitted for the video on the "Assets Upload" page, the replaced subtitles and their corresponding translation results will be displayed here.
Subtitle Dubbing: This module displays the translated subtitles and voice-over results for the video. The system automatically detects speakers (roles) and dialogue content (subtitles) in the video and translates subtitles into the specified language.
Note: The operations for "Text Translation" "Subtitle Translation" and "Text Replacement" are the same as in the "Layout Template" translation section. Below, we will expand on the operations for "Subtitle Dubbing".
π‘By clicking on any text box, the preview window on the right will switch to the position of the corresponding subtitle and its associated language.
You can check if the sentence breaks in the original language text need adjustment. If adjusted, all languages under that subtitle will be modified accordingly.
AI Functionality: You can use AI tools to enrich or trim the original language text. It also supports entering custom commands in the text box, such as limiting the text to 10 characters.
You can split long subtitles into shorter ones by clicking at the desired position. Similarly, you can merge short subtitles into longer ones (supports upward and downward merging).
You can add new subtitle slots or delete the current subtitle.
If you modify the content of the original language subtitle, you need to update the translation results for other languages. This can be done by clicking "Retranslate All Languages" or using the "Update Translation" option at the top to batch update the translation results for other languages.
Update Translation: Updates for all subtitles and languages.
Retranslate All Languages: Updates for a single subtitle and all languages.
If you only modify the translation content of another language's subtitle, you can use "Retranslate" to update it.
Retranslate: Updates for a single subtitle and a specific language.
You can manually fine-tune the translation results by selecting the subtitle text box. AI tools can also be used to process the text.
Clicking the symbol button allows you to modify the subtitle style and adjust its position and size.
You can select different fonts for different languages (if there are square-box characters for less common languages, you can upload the corresponding font package from your local device).
π‘ This text style will be applied synchronously to all languages.
The system automatically matches the "Speaker" and "Gender" for each subtitle segment based on the speakers in the video.
You can manually add speakers, modify speakers, and change gender.
For the same subtitle segment, all languages share the same speaker setting. The same speaker in the same language will have the same voice.
Example: If the video features a single narrator, you can set all original language subtitles to "Speaker 1." If it's a dialogue between two people, you can sequentially set them as "Speaker 1" and "Speaker 2," and choose different voices accordingly.
The dub voice currently includes three categories: Quick Clone, My Voices, and Public Voices. You can choose based on your actual needs.
a.Public Voices
The system provides a rich set of AI voices for commercial use, and you can freely select from them (click the button to listen and preview).
b.Quick Clone
Intelligently copies the voice and tone of the speaker from the original video, making the reading effect more realistic and natural.
c.My Voices
Clone custom voices for characters to achieve more lifelike dubbing effects.
π Jump to see detailed operation steps: Clone Voice Assets
If you modify the subtitle content, speakers, or voices, you need to update the voice-over to apply the changes. This can be done by clicking "Redub" or using the "Dub" option at the top to batch update the voice-over results for other languages.
Dub: Updates the voice-over for all subtitles.
Redub All Languages: Updates the voice-over for a single subtitle and all languages.
Redub: Updates the voice-over for a single subtitle and a specific language.
π‘ To ensure that the voice-over synchronizes with the video after translation, the voice speed of the translated language will be automatically adjusted to match the duration of the original language subtitles. Abnormally slow or fast speed changes may affect the video playback. You can update the voice-over by modifying the text length of the subtitles.
Tips: AI tools can be used to quickly trim or enrich the content!
Preview: After making all the modifications to subtitles and voice-overs, you can click on the designated location to switch and preview the effects for each language and size.
Export
Once you have confirmed the content is correct, click on the "Export" button in the top-right corner to proceed with export settings.
It will display the total number of videos to be exported: Video Variations = Number of material combinations added on the material upload page * Number of project ratios * Number of project languages.
Select the desired size, language, resolution, and bitrate, then check them and click on "Export".
After submitting the export task, you can check the progress of historical tasks under "My Projects >>> Export & Download History >>> Video Projects" list.
Upon successful export, you can either download all the videos to your local device with a single click or directly apply them to authorized advertising channel accounts.
You must be the rightful owner of the files you upload to the Platform. You are responsible for all files you upload to and generate on the platform, and for the consequences of publishing those files elsewhere. If you are unsure about whether you can publish content featuring a cloned voice, we advise that you consider your domestic lawβs stance on copyright before publishing any such content. At Eleven, weβre fully committed both to respecting intellectual property rights and to implementing safeguards against the potential misuse of our technology:
We only partner with clients who adhere to our Terms of Service which prohibit malicious use of our technology towards any purpose which can be deemed illegal or harmful;
When we use recognizable voices, we do so for technology demonstration purposes, on short fragments of works that are either open-source or belong to the public domain, and in contexts which are so far as possible socially and politically neutral. At the same time, all our demos contain clear information about the artificial origin of presented content;
We seek to support voice owners and their licensors in claiming their rights and all known infringements will be reviewed and actioned.
All clients who inquire about professional-grade voice cloning services must secure permission from the voice owner(s) for such use.
All audio generated by our model will be watermarked, so that it can be instantly traced back to us;
The technology weβre developing is new and clear regulation is yet to be introduced. Part of our goal as an AI research lab is to spread awareness about the existence of this technology, its potential, as well as its limitations.
