1

I would like to convert csr_matrix into the SparseTensor. Content of this csr_matrix is stored in npz file. While converting csr_matrix to SparseTensor, I am getting the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/miniconda3/envs/azmldemo/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py in merge_with(self, other)
    947       try:
--> 948         self.assert_same_rank(other)
    949         new_dims = [

~/miniconda3/envs/azmldemo/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py in assert_same_rank(self, other)
   1000       if self.rank != other.rank:
-> 1001         raise ValueError("Shapes %s and %s must have the same rank" %
   1002                          (self, other))

ValueError: Shapes (23373269,) and (None, None) must have the same rank

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
~/miniconda3/envs/azmldemo/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py in with_rank(self, rank)
   1031     try:
-> 1032       return self.merge_with(unknown_shape(rank=rank))
   1033     except ValueError:

~/miniconda3/envs/azmldemo/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py in merge_with(self, other)
    954       except ValueError:
--> 955         raise ValueError("Shapes %s and %s are not compatible" % (self, other))
    956 

ValueError: Shapes (23373269,) and (None, None) are not compatible

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-99-d658fb378c5b> in <module>
----> 1 x_train_tf = tf.sparse.SparseTensor(indices=x_train.indices, values=x_train.data, dense_shape=list(x_train.shape))
      2 x_train_tf

~/miniconda3/envs/azmldemo/lib/python3.8/site-packages/tensorflow/python/framework/sparse_tensor.py in __init__(self, indices, values, dense_shape)
    142     self._dense_shape_default = dense_shape_default
    143 
--> 144     indices_shape = indices.shape.with_rank(2)
    145     values_shape = values.shape.with_rank(1)
    146     dense_shape_shape = dense_shape.shape.with_rank(1)

~/miniconda3/envs/azmldemo/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py in with_rank(self, rank)
   1032       return self.merge_with(unknown_shape(rank=rank))
   1033     except ValueError:
-> 1034       raise ValueError("Shape %s must have rank %d" % (self, rank))
   1035 
   1036   def with_rank_at_least(self, rank):

ValueError: Shape (23373269,) must have rank 2

My code snippet looks like following:

x_train = os.path.join('/home/gaurav.gupta/myproject', 'tmp', 'x_train.npz')
x_train = load_npz(x_train)
x_train_tf = tf.sparse.SparseTensor(indices=x_train.indices, values=x_train.data, dense_shape=list(x_train.shape))

My understanding:

indices must be 2-D array however csr_matrix stores it in 1-D format. For me this boils down to finding the way to represent indices in 2-D format. Any help will be highly appreciated.

Gaurav Gupta
  • 4,586
  • 4
  • 39
  • 72
  • SparseTensor doesn't store data in a sparse row format, so the `indices` from `csr_matrix` are not useful. Try [this answer](https://stackoverflow.com/questions/40896157/scipy-sparse-csr-matrix-to-tensorflow-sparsetensor-mini-batch-gradient-descent). – CJR May 06 '21 at 17:36
  • @CJR This was helpful. – Gaurav Gupta May 06 '21 at 17:51

0 Answers0