3

I saw this question: Implementing custom loss function in keras with condition And I need to do the same thing but with code that seems to need loops.

I have a custom numpy function which calculates the mean Euclid distance from the mean vector. I wrote this based on the paper https://arxiv.org/pdf/1801.05365.pdf:

Equation Picture

import numpy as np

def mean_euclid_distance_from_mean_vector(n_vectors):

    dists = []

    for (i, v) in enumerate(n_vectors):
        n_vectors_rest = n_vectors[np.arange(len(n_vectors)) != i]

        print("rest of vectors: ")
        print(n_vectors_rest)

        # calculate mean vector
        mean_rest = n_vectors_rest.mean(axis=0)

        print("mean rest vector")
        print(mean_rest)

        dist = v - mean_rest

        print("dist vector")
        print(dist)
        dists.append(dist)

    # dists is now a matrix of distance vectors (distance from the mean vector)
    dists = np.array(dists)

    print("distance vector matrix")
    print(dists)

    # here we matmult each vector
    # sum them up
    # and divide by the total number of elements
    result = np.sum([np.matmul(d, d) for d in dists]) / dists.size

    return result


features = np.array([
    [1,2,3,4],
    [4,3,2,1]
])

c = mean_euclid_distance_from_mean_vector(features)

print(c)

I need this function however to work inside tensorflow with Keras. So a custom lambda https://www.tensorflow.org/api_docs/python/tf/keras/layers/Lambda

However, I'm not sure how to implement the above in Keras/Tensorflow since it has loops, and the way the paper talked about calculating the m_i seems to require loops like the way I implemented the above.

For reference, the PyTorch version of this code is here: https://github.com/PramuPerera/DeepOneClass

CMCDragonkai
  • 6,222
  • 12
  • 56
  • 98

1 Answers1

5

Given a feature map like:

features = np.array([
    [1, 2, 3, 4],
    [2, 4, 4, 3],
    [3, 2, 1, 4],
], dtype=np.float64)

reflecting a batch_size of

batch_size = features.shape[0]

and 

k = features.shape[1]

One has that implementing the above Formulas in Tensorflow could be expressed (prototyped) by:

dim = (batch_size, features.shape[1])
def zero(i):
    arr = np.ones(dim)
    arr[i] = 0
    return arr


mapper = [zero(i) for i in range(batch_size)]
elems = (features, mapper)
m = (1 / (batch_size - 1)) * tf.map_fn(lambda x: tf.math.reduce_sum(x[0] * x[1], axis=0), elems, dtype=tf.float64)
pairs = tf.map_fn(lambda x: tf.concat(x, axis=0) , tf.stack([features, m], 1), dtype=tf.float64)
compactness_loss = (1 / (batch_size * k)) * tf.map_fn(lambda x: tf.math.reduce_euclidean_norm(x), pairs, dtype=tf.float64)

with tf.Session() as sess:
    print("loss value output is: ", compactness_loss.eval())

Which yields:

loss value output is:  [0.64549722 0.79056942 0.64549722]

However a single measure is required for the batch, therefore it is necessary to reduce it; by the summation of all values.

The wanted Compactness Loss function à la Tensorflow is:

def compactness_loss(actual, features):
    features = Flatten()(features)
    k = 7 * 7 * 512
    dim = (batch_size, k)

    def zero(i):
        z = tf.zeros((1, dim[1]), dtype=tf.dtypes.float32)
        o = tf.ones((1, dim[1]), dtype=tf.dtypes.float32)
        arr = []
        for k in range(dim[0]):
            arr.append(o if k != i else z)
        res = tf.concat(arr, axis=0)
        return res

    masks = [zero(i) for i in range(batch_size)]
    m = (1 / (batch_size - 1)) * tf.map_fn(
        # row-wise summation
        lambda mask: tf.math.reduce_sum(features * mask, axis=0),
        masks,
        dtype=tf.float32,
    )
    dists = features - m
    sqrd_dists = tf.pow(dists, 2)
    red_dists = tf.math.reduce_sum(sqrd_dists, axis=1)
    compact_loss = (1 / (batch_size * k)) * tf.math.reduce_sum(red_dists)
    return compact_loss

Of course the Flatten() could be moved back into the model for convenience and the k could be derived directly from the feature map; this answers your question. You may just have some trouble finding out the the expected values for the model are - feature maps from the VGG16 (or any other architechture) trained against the imagenet for instance?

The paper says:

In our formulation (shown in Figure 2 (e)), starting froma pre-trained deep model, we freeze initial features (gs) and learn (gl) and (hc). Based on the output of the classification sub-network (hc), two losses compactness loss and descriptiveness loss are evaluated. These two losses, introduced in the subsequent sections, are used to assess the quality of the learned deep feature. We use the provided one-class dataset to calculate the compactness loss. An external multi-class reference dataset is used to evaluate the descriptiveness loss.As shown in Figure 3, weights of gl and hc are learned in the proposed method through back-propagation from the composite loss. Once training is converged, system shown in setup in Figure 2(d) is used to perform classification where the resulting model is used as the pre-trained model.

then looking at the "Framework" backbone here plus:

AlexNet Binary and VGG16 Binary (Baseline). A binary CNN is trained by having ImageNet samples and one-class image samples as the two classes using AlexNet andVGG16 architectures, respectively. Testing is performed using k-nearest neighbor, One-class SVM [43], Isolation Forest [3]and Gaussian Mixture Model [3] classifiers.

Makes me wonder whether it would not be reasonable to add suggested the dense layers to both the Secondary and the Reference Networks to a single class output (Sigmoid) or even and binary class output (using Softmax) and using the mean_squared_error as the so called Compactness Loss and binary_cross_entropy as the Descriptveness Loss.

Índio
  • 539
  • 5
  • 12