How to synchronize mp3 audio with text?

Question

Read all the previous answers and had no luck. Working on an android app in which I need to synchronize mp3 audio with text. As the mp3 played it keeps on changing the text. Just like the lyrics highlights on YouTube etc.

score 3 · Answer 1 · answered Oct 06 '17 at 05:14

first you need to create subtitle file of the audio that you want to play and then add that subtitle file in your media player as

String mimeType = getMimeType("file://mnt/sdcard/BarbieGirl.srt");
        //  mp.selectTrack(index);  
        try {
            mp.addTimedTextSource(path, mimeType);
        } catch (IllegalArgumentException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IllegalStateException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

for creating subtitle file you can use check this url Create Subtitle .

Yes, you can show subtitle on textview follow this link : https://stackoverflow.com/questions/13422673/looking-for-a-working-example-of-addtimedtextsource-for-adding-subtitle-to-a-vid — Ashish, Oct 06 '17 at 06:25

score 1 · Answer 2 · answered Oct 06 '17 at 05:09

They use LRC files to generate the lyrics under the mp3

https://en.wikipedia.org/wiki/LRC_(file_format)

Essentially, the lyrics (or whatever) are tagged with times for which they should appear, if the android player you're using is LRC aware, it should automatically overlay the lyrics at the right time intervals, you can even make the lines appear a few words at a time with the extended tagging.

KeitelDOG · Answer 3 · 2018-02-11T17:36:23.190

I don't have any code sample for android but I made something like this with Javascript, so I'm gonna point you out some steps. Take a look at this LRC Maker & Generator, it contains a sample of "My Heart Will Go On" with lyrics synchronization. If it's what you want to achieve then good.

1- Think LRC, which is a text format to handle synchronization for audio file. Each Line holds information on audio or time within brackets, ex:

[ar:Chubby Checker oppure  Beatles, The]
[al:Hits Of The 60's - Vol. 2 – Oldies]
[ti:Let's Twist Again]
[au:Written by Kal Mann / Dave Appell, 1961]
[length: 2:23]

[00:12.00]Naku Penda Piya-Naku Taka Piya-Mpenziwe
[00:15.30]Some more lyrics ...

Check Wikipedia LRC Format, there is Simple Format (Line-by-Line) and Advanced Format (Word-by-Word). I suggest starting with Simple Format.

2- Create Java Class (Activity or your custom Library) to handle info and objects to read, display and write the Simple Format. Your class should have attributes:

ArrayList of Raw text lines ([00:12.00]Naku Penda Piya-Naku Taka Piya-Mpenziwe, ...)
ArrayList of parsed tags ([00.12.00], [00.18.50], ...)
ArrayList of parsed text lines (Naku Penda Piya-Naku Taka Piya-Mpenziwe, ...)
ListView to display Time Format on left and Lyrics Line on right
Number of Lines to display in your ListView
List Item that holds Views like (Time Format TextView, Lyrics Line TexView or EditText if you allow the User to add or edit each line of lyrics in the App)

3- Code the Engine. You will need to use regular expression to extract formatted lines ([00:12.00]Naku Penda Piya-Naku Taka Piya-Mpenziwe) and time tags in each lines ([00.12.00]). You can use them from javascript code example :

var tagRegex = /\[([a-z]+):(.*)\].*/;
var lrcAllRegex = /(\[[0-9.:\[\]]*\])+(.*)/;
var timeRegex = /\[([0-9]+):([0-9.]+)\]/;
var rawLrcArray = rawLrc.split(/[\r\n]/);

tagRegex is for selecting info on Music (Title, Artist, Genre, etc.)

lrcAllRegex is for selecting a line that has LRC Format.

timeRegex is for selecting time format in a full line with LRC format

rwaLrcArray if for splitting whole text to an array of lines. That must not be difficult in Java.

3a- So using the basic process to add multiple items in ListView via ArrayAdapter in Android, read the Text, divide it in array of multiple lines, parse Time Tags, parse Lines and put them into attributes tag and line to the "LRC item" Class that you will use to populate info in ListView.

LRC Item Class should have attributes :

Time Start Tag
Time End Tag
String Line

For each line, the End Tag will be same as Start Tag of next line. Add Event Listener to your Music Player Playing current time. Each time you get the current time, iterate all LRC Items, check id current time is between LRC start time and end time and then change color of Line EditText to highlight this line.

3b- Next The Part you want the most, for synchronization, Add Event listener to both "Time Tag" TextView, and "Line" EditText of LRC Item Layout. Each time a User clicks to a Time Tag TextView, update the LRC Time Tag value and write that new time in [XX:YY.ZZ] format in Time Tag TextView.

When a User clicks to a Line EditText or TextView, then change Music Player to that current time, this will allow User to go back to any Line to correct Time tag of Next Line.

4- Finally, when done, iterate and grab the new informations on each LRC Item class and concatenate each Time Tag ([00:12.00]) next to each Line (Naku Penda Piya-Naku Taka Piya-Mpenziwe) and add New Line "\n" after each line. It's up to you to decide if you wan't to write it to music-file.lrc or just save it into database.

You can view more on the JS File. Credit to Lusaisai Lyricer Package Live Demo that inspired me.

How to synchronize mp3 audio with text?

3 Answers3