Trouble advancing and redrawing MPEG2 frames... Processing2 GSVideo1.0 OSX10.8.5

Question

My project is using Processing core jar and the GSVideo library on a OSX 10.8.5 using Eclipse.

I cannot get GSVideo jump(int frame) or jump(float time) to actually redraw the next frames. The image displayed toggles back and forth between frames when I repeatedly press the RIGHT to advance the frame in the example program below. Because the example below works with *.mov, but not *.mpg video I want to ask if there are any known problems with gstreamer advancing frames in MPEG2 video. Or perhaps something's up with either java-gstreamer or GSVideo?

I'm working with video in MPEG2 format.. And there is no problem just to play and pause the MPEG2. It just seems that movie.jump(frameNum or time) functions are not working. I've started looking for an example of frame stepping using playbin2's seek method.

Here is info about the video I'm trying to jump.

stream 0: type: CODEC_TYPE_VIDEO; codec: CODEC_ID_MPEG2VIDEO; duration: 7717710; start time: 433367; timebase: 1/90000; coder tb: 1001/60000; width: 1920; height: 1080; format: YUV420P; frame-rate: 29.97;

The example code.

import processing.core.*;
import codeanticode.gsvideo.*;

public class FramesTest extends PApplet {

GSPlayer player;
GSMovie movie;
int newFrame = 0;
PFont font;

public void setup() {
  size(320, 240);
  background(0);
  //movie = new GSMovie(this, "station.mov");  // sample works
  movie = new GSMovie(this, "myMovie.mpg");  // mpg does not
  movie.play();
  movie.goToBeginning();
  movie.pause(); 
  textSize(24);
}

public void movieEvent(GSMovie movie) {
    System.out.println("movie"+ movie.frame());
    movie.read();    
}

public void draw() {
  image(movie, 0, 0, width, height);
  fill(240, 20, 30);

  text(movie.frame() + " / " + (movie.length() - 1), 10, 30);
}

public void keyPressed() {
  if (movie.isSeeking()) return;

  if (key == CODED) {
    if (keyCode == LEFT) {
      if (0 < newFrame) newFrame--; 
    } else if (keyCode == RIGHT) {
      if (newFrame < movie.length() - 1) newFrame++;
    }
  } 

  movie.play();
  movie.jump(newFrame); 
  movie.pause();  
  if (movie.available()){
      System.out.println(movie.frame());
      movie.read();
  }

  System.out.println(newFrame);
}

public static void main(String[] args) {
    // TODO Auto-generated method stub
    PApplet.main(new String[] { FramesTest.class.getName() }); //
     }

}

The example code was pulled from here... http://gsvideo.sourceforge.net/examples/Movie/Frames/Frames.pde

I've searched the internet for a few days and this attempted contact with this forum as well... https://sourceforge.net/projects/gsvideo/forums

This post seems similar but my problem is not playing (that's fine). I cannot jump to a specific frame.... GStreamer: Play mpeg2

Many thanks to the SO community for any help I might receive.

Update: To work around the MPEG2 compression issue (described by v.k. below) I am trying to create a gstreamer pipeline to do on-the-fly transcoding to mp4 using either GSVideo Pipeline or with java-gstreamer. The command below works in Ubuntu.

gst-launch-0.10 filesrc location=myMpeg2Video.mpg ! mpegdemux name=demux demux.video_00 ! ffdec_mpeg2video ! queue ! x264enc ! ffdec_h264 ! xvimagesink

But the following GSVideo Pipeline displays an empty gray window :(

pipeline = new GSPipeline(this, "filesrc location=file:/path/movie.mpg ! mpegdemux name=demux demux.video_00  ! ffdec_mpeg2video"); 
pipeline.play();

Maybe this is related to the fact that mpg codecs doesn't store all frames, there are keyframes (whole) and a sequence of partial frames that just stores what have changed from the key frame. I know that FinalCut struggles when you try to edit with mpeg, and moving frame by frame is painful — v.k., Oct 19 '13 at 00:58
I think you're right about that. I did another sample program just using java-gstreamer and I am able to seek to the next 0.5s. `start = secToNanoLong(now) +timePerFrameNanos; res = playbin.seek(1.0, Format.TIME, SeekFlags.FLUSH, SeekType.SET, start, SeekType.NONE, -1);` That's about 15 seek operations before the frame is updated. — Cris Rockwell, Oct 21 '13 at 17:56

score 0 · Accepted Answer · answered Oct 22 '13 at 15:00

0

as v.k. pointed out, seeking is in general not accurate.

One important thing to note is that development on gsvideo has basically stopped. The main elements of it were ported to the built-in video library in Processing 2.0. I did some work in built-in video to try to improve seeking, and the example Frames in Libraries|video|Movie shows how (to try) to jump to specific frames by indicating a time value. Maybe this helps in your case?

Also if you find a more accurate way of doing seeking as you suggest in your last post, I could include that in video library.

answered Oct 22 '13 at 15:00

codeanticode

16

seek is accurate enough on mp4 (transcoded using gst-launch command). It should be possible to create the pipeline shown above. But I'm having trouble with just basic GSPipelines.... this works `pipeline = new GSPipeline(this, "videotestsrc");` ... but this does not `pipeline = new GSPipeline(this, "playbin2 uri=file:/path/station.mov");` Any ideas? – Cris Rockwell Oct 22 '13 at 19:48
After some trial and error, I've decided it's best to use the Processing2 video library. It doesn't support MPEG2. so I needed to transcode the videos ahead of time. I used this pipeline. `gst-launch-0.10 -v uridecodebin uri=file:///home/me/Videos/test-video1.mpg name=d ! queue ! x264enc ! mp4mux name=m ! filesink location=test-video1.mp4` FYI It doesn't mux in audio. I tried to demo a gst-launch pipeline to transcode then play on the fly. But couldn't prevent dropping frames. GSVideo has limited MPEG2 functionality (only play/pause), so best just to use Processing2 video with mp4's – Cris Rockwell Jan 02 '14 at 18:04

Trouble advancing and redrawing MPEG2 frames... Processing2 GSVideo1.0 OSX10.8.5

1 Answers1