block motion compensated encoder - how to deal with block-noise in reference frame?

Question

I am not proficient in any way with video compression, but currently work on a project where i compress residual images using H.264

My question is more of general nature regarding video encoders. As I understand it (as Wikipedia explains it), a block motion compensated encoder divides the current frame into some non-overlapping blocks, for each block it checks in the reference frame where the current block came from, then it calculates the difference of the desired block and the motion-compensated block, then it encodes this residual somehow.

Wikipedia states: "The main disadvantage of block motion compensation is that it introduces discontinuities at the block borders (blocking artifacts). These artifacts appear in the form of sharp horizontal and vertical edges which are easily spotted by the human eye and produce ringing effects (large coefficients in high frequency sub-bands) in the Fourier-related transform used for transform coding of the residual frames."

So, since the reference frame already contains block-artifacts (because it was encoded), and then these block artifacts are shifted into the current block and the residual is calculated, then the high-frequencies created by the block-artifacts at the borders of the shifted block will also show up as discontinuities in the residual. And discontinuities are generally bad for compression.

Does a motion-compensated block encoder somehow deal with this block-artifacts prior to compressing the residual? since it knows exactly how the block was shifted, it knows where the block borders are located and could do something about them before or while encoding the residual, removing/ignoring the unnecessary high frequencies at this place. If something like this is performed in a codec like H.264, can someone explain the mainconcept of how the encoder does this, give some terminology on it and so on? How does the encoder deal with block-artifacts in the reference frame?

Answer 1

@Mat: This is a good question, and very likely an open area for research in video encoding. The short answer is that to my knowledge, video encoders today do not explicitly do anything to deal with the influence of block-artifacts on motion estimation.

It is possible that in cases of fast motion in areas of low detail, if there are prominent block artifacts, the motion vectors may sometimes be multiples of the block size :) or in other words, motion estimation finds the blocks in the previous frame and not the detail of the original image. This is probably quite rare; it requires both fast motion, a fairly feature-less area of the image, and clearly insufficient bitrate. You can probably construct synthetic video sequences that demonstrate this. Whether there is any way that knowledge of block boundaries may help is not clear; if block edges introduce high-frequency components in the residuals of the next frame, then we have to spend bits to erase/correct those artificially introduced components, whether we know that they came from a block-artifact or not... and even then, spending bits on that may be a better decision than using a different reference area/motion vector/block type.

However, modern codecs such as H.264 and VP8 have an in-loop deblocking filter, in other words the decoded frame is deblocked before being used as a reference, which reduces block noise and therefore probably minimizes the effect of the issue that you have correctly identified.