The intention is to estimate in a [3 3] sliding window. 0.5*[(A(i)-A(5))^2] is computed where a(i) is the pixels around center pixel a(5).
The mean of each of these 8 half square differences is stored in the center pixel's location.
To tackle this conv2 and nlfilter were used on a training example matrix as follows.
clc;
close all;
clear all;
A = [1 2 3 4 5 6;5 4 6 3 2 1;2 3 2 1 4 5];
kernel = [-1 -1 -1; -1 8 -1; -1 -1 -1];
outputImage = conv2(A, kernel);
fun = @(x) mean(x(:));
B= nlfilter (outputImage,[3 3],fun);
The initial idea was to calculate the difference and store it in the location of the pixels instead of the center pixel. Then use a sliding window to take mean of those differences and replace the center pixel.
It was apparent that my logic was flawed.
Though i was able to calculate the difference(i first tried to see simple difference was possible for me) i had to deal with data being overwritten. moreover this method will create a matrix larger than the original which is also wrong.