I'm using Tensorflow's new graphics library to apply a steered convolution to a series of meshes. In many cases, you will have a series of meshes that are not the same size and you must zero-pad the smaller ones. According to the documentation, the "sizes" argument of the graph_conv.feature_steered_convolution_layer function takes in an int tensor consisting of the number of non-padded elements of each mesh. For some reason, when this argument is set something other than "None", I get a warning telling me that the sparse array used in the "neighbors" argument is being converted to a dense matrix. This causes my program to run absurdly slowly.
The issue seems to be tied to the way that it calculates gradients. If the optimizer is commented out, the error does not come up.
I read about a similar problem (link below) where the solution to the problem was to use tf.dynamic_partition rather than tf.gather. However, the tf.gather functions, in this case are located within the graph_convolution library. I attempted to make some edits in my copy of the library, but to no avail.
How to deal with UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from absl.testing import parameterized
import numpy as np
import tensorflow as tf
from tensorflow_graphics.nn.layer import graph_convolution as graph_conv
#Number of meshes
N = 2
#Number of spatial dimensions
d = 2
#################################
#Data consists of the vertices of two meshes. The first mesh has 5 vertices and the second has 4.
#Shape of data is (numberOfMeshes,maxNumberofVertices,numberofSpatialDimensions)
#An array containing the actual size of each non-padded mesh
sz = np.array([5,4],dtype=np.int64)
#The maximum number of vertices in a mesh
datav = 5
#Input placeholder for input data (vertices)
V0 = tf.placeholder(dtype=tf.float64,name="V0",shape=(N,datav,d))
#Input Placeholder for labels for classification (For now, I'm just using throw-away data as my labels)
L = tf.placeholder(shape=(N,5,1),dtype=tf.float64)
SZ = tf.placeholder(shape=(N),dtype=tf.int64)
#Input placeholder for the sparse array representing the adjacency matrix shape:(numberOfMeshes,datav,datav)
#The warning is not raised if "SZ" is changed to "None
adj_sp = tf.sparse_placeholder(shape=(SZ.shape[0],datav,datav),dtype=tf.float64,name='SPP')
#The steered graph convolution that is included in Tensorflow's new graphics package
output = graph_conv.feature_steered_convolution_layer(data=V0,neighbors=adj_sp,sizes=SZ,translation_invariant=False,num_weight_matrices=1,num_output_channels=1)
loss = tf.losses.softmax_cross_entropy(L,output, weights=1.0)
optimizer = tf.train.AdamOptimizer(learning_rate=.001).minimize(loss) #Warning not raised if this is commented out
When the above code is run, I get the following warning:
C:\Python37\lib\site-packages\tensorflow\python\ops\gradients_impl.py:110:
UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown
shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
I am starting to think that this might have more to do with the library itself than with this piece of code. I have referenced this posed in GitHub in case it requires updates (or additional documentation) to the library. https://github.com/tensorflow/graphics/issues/13
