I want help in making video collage, tried everything but unsuccessful

Question

I want to make Video Collage in which 2 or more videos should be displayed in one frame and then they can be converted into one Video file. I tried examples but they just add videos at the end of each video to make a long one combine video. Any Help Please

String FILE_PATH = "/storage/sdcard0/testing.mp4";
String FILE_PATH2 = "/storage/sdcard0/testing1.mp4";
String FILE_PATH3 = "/storage/sdcard0/testing2.mp4";
File file1 = new File(FILE_PATH);
File file2 = new File(FILE_PATH2);
File file3 = new File(FILE_PATH3);

private ProgressDialog pDialog;
ImageView img,img2,img3;
MediaMetadataRetriever retriever2 = new MediaMetadataRetriever();
MediaMetadataRetriever retriever3 = new MediaMetadataRetriever();
ArrayList<Bitmap> bitmapArray1 = new ArrayList<Bitmap>();
ArrayList<Bitmap> bitmapArray2 = new ArrayList<Bitmap>();
ArrayList<Bitmap> bitmapArray3 = new ArrayList<Bitmap>();
File ScreenDIR = new File("/sdcard/Screens/");

// have the object build the directory structure, if needed.

double id1=0,id2=0,id3=0;



@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    ScreenDIR.mkdirs();

     img = (ImageView)findViewById(R.id.imageView);
    img2 = (ImageView)findViewById(R.id.imageView2);
    img3 = (ImageView)findViewById(R.id.imageView3);

    new LoadAllProducts().execute();




}

class LoadAllProducts extends AsyncTask<String, String, String> {

    /**
     * Before starting background thread Show Progress Dialog
     * */
    @Override
    protected void onPreExecute() {
        super.onPreExecute();
        pDialog = new ProgressDialog(MainActivity.this);
        pDialog.setMessage("Extracting Frames. Please wait...");
        pDialog.setIndeterminate(false);
        pDialog.setCancelable(false);
        pDialog.show();
    }

    /**
     * getting All products from url
     * */
    protected String doInBackground(String... args) {
        if(file1.exists()){


            for (long i = 0; i < 5000; i += 1000/14) { // lenms - video length in milliseconds

                MediaMetadataRetriever retriever = new MediaMetadataRetriever();
                retriever.setDataSource(file1.toString());

               // Bitmap bitmap = retriever.getFrameAtTime((i*1000/14), MediaMetadataRetriever.OPTION_CLOSEST_SYNC);

                saveBitmapToCahche( getResizedBitmap((retriever.getFrameAtTime((i*1000/14), MediaMetadataRetriever.OPTION_CLOSEST_SYNC)), 500) ,String.valueOf(id1));
                id1++;
                //bitmapArray1.add(bitmap);

               /* File file = new File(ScreenDIR, "sketchpad1" + id1 + ".png");
                FileOutputStream fOut = null;
                try {
                    fOut = new FileOutputStream(file);
                } catch (FileNotFoundException e) {
                    e.printStackTrace();
                }

                //bitmap.compress(Bitmap.CompressFormat.PNG, 30, fOut);

                try {
                    fOut.flush();
                } catch (IOException e) {
                    e.printStackTrace();
                }
                try {
                    fOut.close();


                } catch (IOException e) {
                    e.printStackTrace();
                }*/
            }

            }
       /* if(file2.exists()){
            retriever2.setDataSource(file2.toString());
            for (long i = 0; i < 3000; i += 1000/24) { // lenms - video length in milliseconds
                bitmap2 = retriever2.getFrameAtTime(i*1000/29, MediaMetadataRetriever.OPTION_CLOSEST_SYNC);
                //bitmapArray2.add(bitmap2);

                File file = new File(ScreenDIR, "sketchpad2" + id2 + ".png");
                FileOutputStream fOut = null;
                try {
                    fOut = new FileOutputStream(file);
                } catch (FileNotFoundException e) {
                    e.printStackTrace();
                }

                bitmap2.compress(Bitmap.CompressFormat.PNG, 85, fOut);
                try {
                    fOut.flush();
                } catch (IOException e) {
                    e.printStackTrace();
                }
                try {
                    fOut.close();
                    id2++;
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }



        }
        if(file3.exists()){
            retriever3.setDataSource(file3.toString());
            for (long i = 0; i < 3000; i += 1000/24) { // lenms - video length in milliseconds
                bitmap3 = retriever3.getFrameAtTime(i*1000/29, MediaMetadataRetriever.OPTION_CLOSEST_SYNC);
               // bitmapArray3.add(bitmap3);

                File file = new File(ScreenDIR, "sketchpad3" + id3 + ".png");
                FileOutputStream fOut = null;
                try {
                    fOut = new FileOutputStream(file);
                } catch (FileNotFoundException e) {
                    e.printStackTrace();
                }

                bitmap3.compress(Bitmap.CompressFormat.PNG, 85, fOut);
                try {
                    fOut.flush();
                } catch (IOException e) {
                    e.printStackTrace();
                }
                try {
                    fOut.close();
                    id3++;
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

            }*/




        return null;
    }
    /**
     * After completing background task Dismiss the progress dialog
     * **/
    protected void onPostExecute(String file_url) {
        // dismiss the dialog after getting all products
        pDialog.dismiss();



        img.setImageBitmap(retrieveBitmapFromCache(String.valueOf(id2)));
        id2 = 50;

        img2.setImageBitmap(retrieveBitmapFromCache(String.valueOf(id2)));
        id2 = 69;

        img3.setImageBitmap(retrieveBitmapFromCache(String.valueOf(id2)));


       // img2.setImageBitmap(bitmapArray2.get(0));
       // img3.setImageBitmap(bitmapArray3.get(0));

        }




    }


public void saveBitmapToCahche(Bitmap bb,String ID ){

    Cache.getInstance().getLru().put(ID, bb);


}
public Bitmap retrieveBitmapFromCache(String ID) {


    Bitmap bitmap = (Bitmap) Cache.getInstance().getLru().get(ID);

    return  bitmap;

}
public Bitmap getResizedBitmap(Bitmap image, int maxSize) {
    int width = image.getWidth();
    int height = image.getHeight();

    float bitmapRatio = (float)width / (float) height;
    if (bitmapRatio > 0) {
        width = maxSize;
        height = (int) (width / bitmapRatio);
    } else {
        height = maxSize;
        width = (int) (height * bitmapRatio);
    }
    return Bitmap.createScaledBitmap(image, width, height, true);
}
}

`

Answer 1

Split video into frames, e.g.

MediaMetadataRetriever retriever = new MediaMetadataRetriever();
File file1 = new File(directory, "file.mp4");
retriever.setDataSource(file1.toString());
for (long i = 0; i < lenms; i += 1000/fps) { // lenms - video length in milliseconds
    Bitmap bitmap = retriever.getFrameAtTime(i*1000/fps, MediaMetadataRetriever.OPTION_CLOSEST_SYNC);
}

and make a new frames from bitmap

Answer 2

Creating a bitmap for each frame of video is terribly slow and not storage friendly at all. besides that MediaMetadataRetriever will give you lots of redundant frames. when you are done with creating bitmaps for both videos you will only have 4-5 different bitmaps from both videos.

To extract each and every frame from videos you will have to use MediaCodec with MediaExtractor.

Creating a collage is bit tricky. one thing is H264 coded only supports certain frame sizes. when you add two video side by side it wont necessarily fit into supported frame sizes.

That said, i thinks its possible with rendering frames side by side from both videos onto GLSurfaceView and sharing that surface to Mediacodec which will encode that frames to h264. i have not implemented this but i believe this is one way to do it without dealing with NDK.

FFMPEG also has in built functionality for this. if you are comfortable with NDK.