1

I'm trying to program an video/audio dubbing editor. Trying to mix serveral audio files in vala/genie. Using adder or interleave.

I need traslate this gst-launch command to Genie or Vala code but using: 1.- Gst.Element.link 2.- Gst.ElementFactory.make 3.- request_pad and others... Please don't use parse_launch()

gst-launch-0.10 interleave name=i ! audioconvert ! wavenc ! filesink location=file.wav  filesrc location=file1.wav ! decodebin ! audioconvert ! "audio/x-raw-int,channels=1" ! queue ! i.   filesrc location=file2.wav !  decodebin ! audioconvert ! "audio/x-raw-int,channels=1" ! queue ! i.

I have this Genie code:

[indent=4]

uses
    Gst

pipeline: private Pipeline 
interleave: private Element
audioconvert: private Element
audioconvert2: private Element
audioconvert3: private Element
wavenc: private Element
decodebin2: private Element
decodebin3: private Element
capsfilter2: private Element
capsfilter3: private Element
filesink: private Element
src3:private Element
src2:private Element
queue2:private Element
queue3:private Element
capsfilter2a:private Element
capsfilter3a:private Element

init
    Gst.init (ref args)
    pipeline = new Pipeline ("mypipeline")

    interleave=ElementFactory.make ("interleave","inter")
    interleave.set ("name","i")
    audioconvert=ElementFactory.make ("audioconvert","audioconvert")
    wavenc=ElementFactory.make ("wavenc","wavenc")
    filesink=ElementFactory.make ("filesink","filesink")
    filesink.set("location","totalfinal.wav")

    // primer archivo
    src2=  ElementFactory.make ("filesrc", "filesrc2")
    src2.set ("location","file1.wav")
    decodebin2=  ElementFactory.make ("decodebin", "decodebin2")
    audioconvert2=  ElementFactory.make ("audioconvert", "audioconvert2")
    capsfilter2= ElementFactory.make ("capsfilter","capsfileter2")
    capsfilter2.set("caps", Gst.Caps.from_string("audio/x-raw-int,channels=1"))
    queue2= ElementFactory.make ("queue","queue2")
    capsfilter2a= ElementFactory.make ("capsfilter","capsfileter2a")
    capsfilter2a.set("caps", Gst.Caps.from_string("i.sink_0"))

    // segundo archivo
    src3=  ElementFactory.make ("filesrc", "filesrc3")
    src3.set ("location","file2.wav")
    decodebin3=  ElementFactory.make ("decodebin", "decodebin3")
    audioconvert3=  ElementFactory.make ("audioconvert", "audioconvert3")
    capsfilter3= ElementFactory.make ("capsfilter","capsfileter3")
    capsfilter3.set("caps", Gst.Caps.from_string("audio/x-raw-int,channels=1"))
    queue3= ElementFactory.make ("queue","queue3")
    capsfilter3a= ElementFactory.make ("capsfilter","capsfileter3a")
    capsfilter3a.set("caps", Gst.Caps.from_string("i.sink_1"))


    pipeline.add_many(interleave,wavenc,filesink,audioconvert);
    pipeline.add_many(src2,src3,decodebin2,decodebin3,audioconvert2,audioconvert3,capsfilter2,capsfilter2a,capsfilter3,capsfilter3a);
    pipeline.add_many(queue2,queue3);

    // basic line
    interleave.link(audioconvert)
    audioconvert.link(wavenc)
    wavenc.link(filesink)

    // first src
    src2.link(decodebin2)
    decodebin2.link(audioconvert2)
    audioconvert2.link(capsfilter2)
    capsfilter2.link(queue2)
    queue2.link(capsfilter2a)

    // second src
    src3.link(decodebin3)
    decodebin3.link(audioconvert3)
    audioconvert3.link(capsfilter3)
    capsfilter3.link(queue3)
    queue3.link(capsfilter3a)


    pipeline.set_state(Gst.State.PLAYING)

    new MainLoop().run();

I don't know what is going wrong.

txasatonga
  • 419
  • 2
  • 11
  • I have edited the above code sample so that at least it compiles. If you are not using tabs for indentation, you need to specify it at the top of the file in the ```[indent=x]``` tag where x is the number of spaces per indent. – Che Bizarro Jan 05 '16 at 03:42
  • There were also two semi-colons at the end of the first two lines that I removed which would also cause it to error. I was able to compile and run the above code, but I am not sure what you are expecting as the output. An empty file called totalfinal.wav is generated and the program will keep running until it is terminated with Ctrl+C. What do you expect the final result to be? – Che Bizarro Jan 05 '16 at 06:09
  • thanks Che. I think i should use something like get_request_pad('sink%d') but i dont know how to use. – txasatonga Jan 05 '16 at 10:57
  • I think i should use "added" element and something like get_request_pad('sink%d'). I think each file block should be a bin. See this link :http://freshfoo.com/downloads/trackmix.py – txasatonga Jan 05 '16 at 11:05
  • I have changed my answer with a complete solution in Vala. This should now serve as a basis for an audio mixing application. – Che Bizarro Jan 06 '16 at 09:50

3 Answers3

2

The following Vala code works as long as the files are the same format (.wav or .mp3).

using Gst;

public class AudioFileSource : Gst.Bin {

    string filename;
    Gst.Caps outcaps;
    Gst.Element filesrc;

    Gst.Element dbin; 
    Gst.Element ident;
    Gst.Element audioconvert;
    Gst.Element volume;

    /**
     * AudioFileSource - creates an input source Bin from a filename
     * @param filename: The name of the audio file to load. 
     * @param volume: Volume level (fractional, may go above 1.0)
     * @param outcaps: Optional GStreamer capabilities object. A sensible
        default will be used if not given.
     */ 
    public AudioFileSource (string filename, double volume = 0.5, Gst.Caps? outcaps = null) {

        this.filename = filename;

        if (outcaps == null)
            this.outcaps = Gst.Caps.from_string("audio/x-raw,format=int,channels=2,rate=44100,depth=16");
        else
            this.outcaps = outcaps;

        filesrc = Gst.ElementFactory.make("filesrc", "src");
        filesrc.set("location", this.filename);

        dbin = Gst.ElementFactory.make("decodebin", null);
        ident = Gst.ElementFactory.make("identity", null);
        audioconvert = Gst.ElementFactory.make("audioconvert", null);

        this.volume = Gst.ElementFactory.make("volume", null);
        this.volume.set_property("volume", volume);

        add_many(filesrc, dbin, ident, audioconvert, this.volume);
        filesrc.link(dbin);

        audioconvert.link(this.volume);
        this.volume.link_filtered(ident, outcaps);

        // Create an output from this bin (the "src" of the ident instance
        // becomes the the "src" of this bin)
        var srcpad = new Gst.GhostPad("src", ident.get_static_pad("src"));
        add_pad(srcpad);

        dbin.pad_added.connect((e, p) => {
            p.link(audioconvert.get_static_pad("sink"));
        });
    }
}

public class SimpleMixer : GLib.Object {
    /*
    Simple class that mixes audio sources straight over the top of each other.
    */

    Gst.Pipeline pipeline;
    Gst.Element mixer;
    Gst.Element audioconvert;
    Gst.Element sink;

    AudioFileSource[] sources = {};

    public signal void finished();

    public SimpleMixer () {
        pipeline = new Gst.Pipeline("mypipeline");
        mixer = Gst.ElementFactory.make("adder", null);
    }

    public void stop(Message msg) {
        if (msg.type == Gst.MessageType.EOS) {
            pipeline.set_state(Gst.State.NULL);
            finished();
        }
    }

    public void add_source (string filename) {
        sources += new AudioFileSource(filename);
    }

    public void mix (string output) {

        audioconvert = Gst.ElementFactory.make("wavenc", null);
        sink = Gst.ElementFactory.make("filesink", "sink");
        sink.set("location", output);

        foreach (var source in sources)
            pipeline.add(source);

        pipeline.add(mixer);

        foreach (var source in sources)
            source.get_static_pad("src").link(mixer.get_request_pad("sink_%u"));

        pipeline.add(audioconvert);
        mixer.link(audioconvert);
        pipeline.add(sink);
        audioconvert.link(sink);

        var bus = pipeline.get_bus();
        bus.add_signal_watch(GLib.Priority.HIGH);
        bus.message.connect(stop);

        pipeline.set_state(Gst.State.PLAYING);
    }
}

static int main(string[] args) {
    Gst.init(ref args);
    var mainloop = new MainLoop();

    var mixer = new SimpleMixer();
    for (int i = 2; i < args.length; i++)
        mixer.add_source (args[i]);

    mixer.finished.connect(() => {
        mainloop.quit();
    });

    mixer.mix(args[1]);

    mainloop.run();

    return 0;
}

You should compile this with gstreamer-1.0:

valac --pkg gstreamer-1.0 mixer.vala

To use it:

./mixer totalfinal.wav file1.wav file2.wav

Hope this helps get you started.

Che Bizarro
  • 136
  • 10
  • Ok. Thanks Che. But I would like to know how to link them using Faktory make and linking commands. – txasatonga Dec 31 '15 at 00:46
  • By other way, I those archives aren't unlinkeable and new others cant be linked. Isen't it? – txasatonga Dec 31 '15 at 00:48
  • Do you mean using a Makefile instead of compiling with Vala directly? – Che Bizarro Dec 31 '15 at 01:02
  • ups!! Sorry. No. :-> using this vala methods: 1.- Gst.Element.link 2.- Gst.ElementFactory.make – txasatonga Dec 31 '15 at 02:06
  • Hey!! Hey!!! Che Bizarro!!! is just what i need!! I was working around this!! Thanks. Now I only must add a parser for seeking into the audio, but now i know how. Thanks. By other way, I have search too the posibility of use dinamicaly parse_launch() method. You can give a name to a element into the string and after take the element out using .get_by_name(element_name) method. But I thing, your answer is more perfect. Thanks. – txasatonga Jan 06 '16 at 20:30
1

Thanks to Che Bizarro I have solved with this simple code in Genie.

uses Gst
//adder name=mix ! volume volume=0.1 ! alsasink filesrc location=file1.wav ! wavparse ! audioconvert ! mix. filesrc location=file2.wav ! wavparse ! audioconvert ! mix. filesrc location=file3.wav ! wavparse ! audioconvert !  mix.


class  AudioFilesSrc : Gst.Bin
    wavparse: Element
    src:Element
    audioconvert: Element

    def OnDynamicPad (element:Element, zz:Pad)
        var opad = audioconvert.get_static_pad("sink");
        zz.link(opad);

    def open(s1:string)

        src = Gst.ElementFactory.make("filesrc", "src1");
        wavparse = ElementFactory.make("wavparse","wavparse");
        audioconvert = ElementFactory.make("audioconvert","audioconvert");
        wavparse.pad_added.connect(OnDynamicPad);
        this.add_many(src,wavparse,audioconvert);
        src.link_many(wavparse,audioconvert);
        src.set("location",s1);




class Pipe:GLib.Object
    pipeline:Pipeline
    adder: Element
    sink: Element


    def create()    
        pipeline= new Pipeline("test");
        adder = ElementFactory.make("adder","mixer");
        var volume= ElementFactory.make("volume","volume");
        volume.set("volume",0.1)
        sink= ElementFactory.make("autoaudiosink","alsasink");
        pipeline.add_many(adder,volume,sink)
        adder.link_many(volume,sink)

    def conector(bin:Bin)
        // Add the new Bin to our pipeline.
        pipeline.add(bin)
        // search the audioconvert element and get its src for give it to the Bin.
        var e=bin.get_by_name("audioconvert")
        var srcpad = new Gst.GhostPad("src", e.get_static_pad("src"));
        bin.add_pad(srcpad);
        // create a new sink pad into the adder element
        var ch1_sinkpad = adder.get_request_pad("sink%d")
        // Link sources to mixer
        bin.get_pad("src").link(ch1_sinkpad)

    def play()
        pipeline.set_state(Gst.State.PLAYING);



init
    Gst.init(ref args);
    var pipe= new Pipe()
    pipe.create()

    var archivos= new list of AudioFilesSrc
    for var i=0 to 1
        archivos.add(new AudioFilesSrc())
        archivos[i].open("file"+(i).to_string()+".wav") // you need several filenames with this format file%d.wav : file0.wav, file1.wav....
        pipe.conector(archivos[i])
    pipe.play()

    new MainLoop().run();
txasatonga
  • 419
  • 2
  • 11
0

There are a lot of details to consider when manually building a pipeline, specially when linking elements together. I recommend you using the available parser wrappers, which are way more convenient. Take a look at the following snippet, it does exactly the same as yours, but takes care of the details internally (and looks cleaner to me)

Good luck! Gruner

uses Gst

pipeline: private Pipeline 
gst_launch: private string

init
    Gst.init (ref args);

    gst_launch = "interleave name=i ! audioconvert ! wavenc ! filesink location=file.wav filesrc location=file1.wav ! decodebin ! audioconvert ! 'audio/x-raw-int,channels=1' ! queue ! i.   filesrc location=file2.wav ! decodebin ! audioconvert ! 'audio/x-raw-int,channels=1' ! queue ! i."

    pipeline = Gst.parse_launch(gst_launch) as Pipeline;

    pipeline.set_state(Gst.State.PLAYING)

    new MainLoop().run();
  • Thanks, but I think that if I use parse_launch("") I can't control the seek of the pipeline. isn't it? – txasatonga Jan 05 '16 at 10:55
  • I know how take out an element who was named in the gst_launch string. Therefore your answer will be other way, but don't answer directly what I need. Thanks for your attention. – txasatonga Jan 06 '16 at 20:36