2

I am working on a video editing application.

I need a function to do chroma keying and replace green background in a video file (not live feed) with an image or another video.

I have looked through the GPUImage framework but it is not suitable for my project as I cannot use third party frameworks for this project so I am wondering if there is another way to accomplish this.

Here are my questions:

  • Does it need to be done through Shaders in Opengl es?
  • Is there another way to access the frames and replace the background doing chroma keying using the AV Foundation framework?

I am not too versed in Graphics processing so I really appreciate any help.

Julien Vavasseur
  • 3,854
  • 1
  • 20
  • 29
JpHerron
  • 21
  • 1
  • 3
  • I would recommend looking into CoreGraphics. I haven't done the image processing on the scale you are referring to, but you can use CoreGraphicsContexts to edit UIImages that you pass into it, and get UIImages back out (you can get the pixel colors, and set pixels, draw, etc.). However, that would run on the CPU, which depending on the video size, may not work for you. – Gliderman Nov 12 '15 at 21:09
  • 2
    Why "can't you use 3rd party" software? I am not saying Brad's green screen approach is actually the perfect solution (there are serious issues with green screen logic) but why would you start out by not using existing known working 3rd party solutions? Video software is really complex and one should not try to reinvent the wheel without a very good reason. – MoDJ Nov 12 '15 at 22:01
  • Check out the [GPUImage2](https://github.com/BradLarson/GPUImage2) project. It's written in Swift 3 and supports chroma keying. – Felix Dec 06 '17 at 13:44

1 Answers1

7

Apple's image and video processing framework is CoreImage. There is sample code for Core Image which includes a ChromeKey example.

No it's not in Swift, which you asked for. But as @MoDJ says; video software is really complex, reuse what already exists. I would use Apple's Obj-c sample code to get something that works then, if you feel you MUST have it in Swift, port it little by little and if you have specific problems ask them here.

alexkent
  • 1,556
  • 11
  • 25
  • There's now a dropdown in the upper left of the linked website that lets you select Swift. Unfortunately, the triple for loop for the color cube makes it really expensive for your phone to pull off in real time. – Chewie The Chorkie Dec 19 '18 at 19:22