1

I'm trying to modify the motion detection part of FFMPEG. What I want to do is to extend the search space, so that whenever the macroblock hit the right most edge of the frame, I need it to still move the block towards the left-most as if they are connected (in my example videos, the right edge is actually a continue of the left edge). Can someone help me to point where exactly I can modify it within FFMPEG source code or x265, or x264?

enter image description here

I took H265 as an example from here. It has a motion.cpp file which nicely specifies the possible block sizes as below. But I can't find the specific loop that traverses the frame. A help is highly appreciated.

#define SETUP_SCALE(W, H) \
    sizeScale[LUMA_ ## W ## x ## H] = (H * H) >> 4;
    SETUP_SCALE(4, 4);
    SETUP_SCALE(8, 8);
    SETUP_SCALE(8, 4);
    SETUP_SCALE(4, 8);
    SETUP_SCALE(16, 16);
    SETUP_SCALE(16, 8);
    SETUP_SCALE(8, 16);
    SETUP_SCALE(16, 12);
    SETUP_SCALE(12, 16);
    SETUP_SCALE(4, 16);
    SETUP_SCALE(16, 4);
    SETUP_SCALE(32, 32);
    SETUP_SCALE(32, 16);
    SETUP_SCALE(16, 32);
    SETUP_SCALE(32, 24);
    SETUP_SCALE(24, 32);
    SETUP_SCALE(32, 8);
    SETUP_SCALE(8, 32);
    SETUP_SCALE(64, 64);
    SETUP_SCALE(64, 32);
    SETUP_SCALE(32, 64);
    SETUP_SCALE(64, 48);
    SETUP_SCALE(48, 64);
    SETUP_SCALE(64, 16);
    SETUP_SCALE(16, 64);
#undef SETUP_SCALE

UPDATE:

One way to do that (in x265) is to modify the edge extension area (which is already in the code, in the frameFilter.cpp), and do that for rightmost and fill blocks with leftmost pixels. I identified the piece of code here. Can someone help me to add this feature for right-to-left extension?

if ((col == 0) | (col == m_frameFilter->m_numCols - 1))
    {
        // TODO: improve by process on Left or Right only
        primitives.extendRowBorder(reconPic->getLumaAddr(m_rowAddr), stride, reconPic->m_picWidth, realH, reconPic->m_lumaMarginX);

        if (m_frameFilter->m_param->internalCsp != X265_CSP_I400)
        {
            primitives.extendRowBorder(reconPic->getCbAddr(m_rowAddr), strideC, reconPic->m_picWidth >> hChromaShift, realH >> vChromaShift, reconPic->m_chromaMarginX);
            primitives.extendRowBorder(reconPic->getCrAddr(m_rowAddr), strideC, reconPic->m_picWidth >> hChromaShift, realH >> vChromaShift, reconPic->m_chromaMarginX);
        }
    }

    // Extra Left and Right border on first and last CU
    if ((col == 0) | (col == m_frameFilter->m_numCols - 1))
    {
        copySizeY += lumaMarginX;
        copySizeC += chromaMarginX;
    }

    // First column need extension left padding area and first CU
    if (col == 0)
    {
        pixY -= lumaMarginX;
        pixU -= chromaMarginX;
        pixV -= chromaMarginX;
    }
Tina J
  • 4,983
  • 13
  • 59
  • 125
  • 1
    this is an uneducated and possibly useless suggestion, not having worked with this system before, and I apologize if I'm misunderstanding your question, but other times I've needed to do something similar to this I've just created a second box located mostly off-screen to the left, in addition to the original, whenever the original one starts to go off-screen to the right. So as one moves off-screen, the other moves on-screen. – HammerN'Songs Jun 27 '16 at 15:35
  • Yeah, it seems similar to what I want. Did you work on top of libx265 code? I just need to fill in the off-screen parts of the box with the pixles in the left-most. I think if I just add `%` in the index it should be enough. Right? – Tina J Jun 27 '16 at 18:27
  • I'm not sure how the box works, I haven't actually worked with any of this. I've just worked with other systems that needed to wrap objects/regions across borders. If you can find a loop where the box iterates through pixels, looking for the ones which lie beneath it, then you should be able to just add in a % in all the places where the box tries to interact with the background. I'm not sure there's a place to do that though. But looking at the code, if I had to guess what the snippet you posted did, I would have guessed that it deals with what you're asking already. (1/2) – HammerN'Songs Jun 27 '16 at 19:09
  • I can't find where extendRowBorder is defined, but I would guess it copies the image and pastes it next to itself, so that as the search box moves of the original image, it will move onto the left side of the copy, giving the same results as if the box just wrapped around the original image. That appears to be exactly what happens when the box hits the top or bottom, a couple lines below the snippet you posted, but if the side-to-side 'wrapping' doesn't work, then I'd try to find what happens in extendRowBorder(). It's declared in primitives.h, but I can't find where it's defined. (2/2) – HammerN'Songs Jun 27 '16 at 19:16
  • They have an `if` for when the edge is hit. But they don't do what I want. So the part that fills the off-screen parts is missing. – Tina J Jun 27 '16 at 19:21
  • CU is the search block, right? – Tina J Jun 27 '16 at 19:22
  • I think so, I'm not sure. And I misread the code, extendRowBorder is just a type, not a function. So I might try copying the "border extended top" and "border extended bottom" sections, and changing the copies to work with left/right respectively, instead of top/bottom, just to see if that works. With and without the snippet you posted commented out. I'd be careful though. Or it might be that the input isn't what it's expecting? I can't figure out what the inner nested if-statement is checking, in the snippet you posted, or why it runs the same line of code twice. Sorry I'm not more helpful. – HammerN'Songs Jun 27 '16 at 19:40
  • I didn't get why I should copy the cod for top and bottom into left and right. Top and bottom are also not implemented I guess?! And those two lines are different: one for Cb and one for Cr. – Tina J Jun 27 '16 at 21:35
  • You're right, I missed that. But I guess I should have asked first - does it not do what you want when the box goes off-screen on the top/bottom? – HammerN'Songs Jun 28 '16 at 13:10
  • Ummm, you mean they do what I want for up to down, not right-to-left?! There is a code for up to down right after left to right, but I'm not sure if they do it. Can you see here? http://pastebin.com/dX1SH0pe – Tina J Jun 28 '16 at 23:53
  • 1
    That's the code I meant you might be able to copy and alter. It appears what they do is copy the background above or below itself, which might mean what you want would work for the top and bottom, but I'm not sure if it actually works. You'd need to test it, to see if scrolling up to down works, and if it does, then you might be able to just copy the up to down section and change it to do the same thing, just left to right. Maybe. – HammerN'Songs Jun 29 '16 at 15:50
  • Yeah, I notice that before. Thanks for the hint. I'm just wondering why they didn't really do that for right to left :-/ I remember a bit upper in the code they said TODO feature...Probably they just extend the border, but not copy them. But the code says it copies! ...I'll add that and will let you know (I've never even built x265 before!) – Tina J Jul 01 '16 at 06:17

0 Answers0