3

I would like to copy pixels from a 1080p video from one location to another efficiently/with as little CPU impact as possible.

So far my implementation is fairly simple:

  • using BitmapData's draw() method to grab the pixels from the video
  • using BitmapData's copyPixels() to shuffle pixels about

Ideally this would have as little CPU impact as possible but I am running out of options and could really use some tips from experienced actionscript 3 developers.

I've profiled my code with Scout and noticed the CPU usage is mostly around 70% but goes above 100% quite a bit. I've looked into StageVideo but one of the main limitations is this:

The video data cannot be copied into a BitmapData object (BitmapData.draw).

Is there a more direct way to access video pixels, rather than rasterizing a DisplayObject ?

Can I access each video frame as a ByteArray directly and plug it into a BitmapData object ? (I found appendBytes but it seems to do the reverse of what I need in my setup).

What is the most CPU friendly way to manipulate pixels from an h264 1080p video in actionscript 3 ?

Also, is there a faster way to moving pixels around other than copyPixels() using Flash Player ?Also, I see Scout points out that video is not hardware accelerated( .rend.video.hwrender: false ). Shouldn't h264 video be hardware accelerated (even without stage video) according to this article (or is this for the fullscreen mode only) ?

George Profenza
  • 50,687
  • 19
  • 144
  • 218
  • 1
    I'd say manipulating this large set of data with several layers of abstraction should not get faster than what you already have, and yes, `copyPixels()` is one of the fastest methods there is, probably you can get a tad faster with `getPixels()` or `getVector()`, depending on what do you actually need to do with them. – Vesper Oct 09 '14 at 10:35
  • Thank you for clearing the info on copyPixels(). Do you know any methods of fetching pixels from video directly (without having to use use BitmapData'draw()) ? – George Profenza Oct 09 '14 at 11:25
  • Don't bother with grabbing bytes. There won't be any pixels until you decode from compression (ie: write your own custom h.264 decoder). If you use appendBytes as a decoding shortcut then yes the bytes become pixels but guess what? `BitmapData.draw()` is blocked by Adobe!! Where playing the file could draw, the _exact same or any custom video data_ fed via byteArray throws a policy file error (wtf? though I can understand why they did it) – VC.One Oct 15 '14 at 08:59
  • You can try converting h.264 to h.263 (in FLV container) and embed into SWF, then you can load that and draw/effect frame-by-frame. Don't do in realtime but instead buffer at least a few seconds worth of bitmapdata into some byteArray then append. – VC.One Oct 15 '14 at 09:08
  • @VC.One Thank you for the hacky embedded idea, but I will need to use dynamic content. The custom decoder option sounds interesting, perhaps something can be done with alchemy/CrossBridge and keep most of it on the GPU, but it feels a bit overkill – George Profenza Oct 16 '14 at 23:27

1 Answers1

2

Latest AIR beta introduced video as texture support which you could possibly use to manipulate the video on GPU (and do that way faster than with BitmapData). But keep in mind that it is currently available for AIR on Windows only and there are some other limitations.

Varnius
  • 1,527
  • 1
  • 12
  • 14
  • Thank you for the suggestion(+1), I didn't know that. However I need to use the Flash Player runtime, not AIR and support as many desktop systems as possible. Would you happen to know a similar method for Flash Player ? – George Profenza Oct 09 '14 at 13:08
  • Not really, afaik there is no other way to efficiently upload a frame to GPU without resorting to BitmapData. But I'm pretty sure VideoTexture support will come to Flash Player in the future, so until then you should stick to BitmapData approach. – Varnius Oct 09 '14 at 13:41