1

I am a new to tensorflow, I am trying to build a simple neural network. But every time I get close, there are a list of errors stopping me. I followed tutorials and documentations and kept most of the code and changed only things I need to.

Here is my code:

###
# Import
###

import tensorflow as tf
import pandas as pd
from tensorflow import keras

###
# Loading Data
###


# Training Data

# path of the training data
train_data_path = "C:/Users/User/Desktop/Machine_Learning/Neural_Network/addition_train_data.csv"
train_data = pd.read_csv(train_data_path)  # loads the data using pandas

# Evalution Data

# path of the evalution data
eval_data_path = "C:/Users/User/Desktop/Machine_Learning/Neural_Network/addition_eval_data.csv"
eval_data = pd.read_csv(eval_data_path)  # loads the data using pandas (again)


# Target Columns
train_target = train_data.pop("Sum")
eval_target = eval_data.pop("Sum")


###
# Creating the Model
###

model = keras.Sequential()
model.add(keras.layers.Flatten(input_shape=(35, 2)))
model.add(keras.layers.Lambda(
    lambda x: tf.expand_dims(model.output, axis=-1)))
model.add(keras.layers.Dense(10, activation="tanh"))
model.add(keras.layers.Dense(1, activation="tanh"))

###
# Compiling the Model
###

model.compile(
    optimizer='adam',
    loss='mean_absolute_error',
    metrics=['accuracy']
)

###
# Training the Model
###

model.fit(
    eval_data, eval_target, epochs=10
)

Console Output:

2020-05-25 10:53:35.491127: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not 
load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found                                                           
2020-05-25 10:53:35.493137: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart 
dlerror if you do not have a GPU set up on your machine.                                                                                
2020-05-25 10:53:37.162913: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
opened dynamic library nvcuda.dll                                                                                                   
2020-05-25 10:53:37.194951: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with 
properties:                                                                                                                         
pciBusID: 0000:01:00.0 name: GeForce RTX 2060 computeCapability: 7.5                                                                                                                                                                         
coreClock: 1.755GHz coreCount: 30 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 312.97GiB/s                                                                                                                                               
2020-05-25 10:53:37.200604: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not 
load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found                                                           
2020-05-25 10:53:37.206365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
opened dynamic library cublas64_10.dll                                                                                              
2020-05-25 10:53:37.212086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
opened dynamic library cufft64_10.dll                                                                                               
2020-05-25 10:53:37.214531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
opened dynamic library curand64_10.dll                                                                                              
2020-05-25 10:53:37.219340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
opened dynamic library cusolver64_10.dll                                                                                            
2020-05-25 10:53:37.224932: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
opened dynamic library cusparse64_10.dll                                                                                            
2020-05-25 10:53:37.233220: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
opened dynamic library cudnn64_7.dll                                                                                                
2020-05-25 10:53:37.235711: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1598] Cannot dlopen some 
GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would 
like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup 
the required libraries for your platform.                                                                                                                     
Skipping registering GPU devices...                                                                                                                                                                                                          
2020-05-25 10:53:37.241553: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports 
instructions that this TensorFlow binary was not compiled to use: AVX2                                                                    
2020-05-25 10:53:37.249295: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1d9e0a5c5f0 
initialized for platform Host (this does not guarantee that XLA will be used). Devices:                                              
2020-05-25 10:53:37.252889: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device 
(0): Host, Default Version                                                                                                             
2020-05-25 10:53:37.255179: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect 
StreamExecutor with strength 1 edge matrix:                                                                                         
2020-05-25 10:53:37.258151: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]                                                                                                                                                         
Epoch 1/10                                                                                                                                                                                                                                   
WARNING:tensorflow:Model was constructed with shape (None, 35, 2) for input Tensor("flatten_input:0", 
shape=(None, 35, 2), dtype=float32), but it was called on an input with incompatible shape (None, 2).                                  
WARNING:tensorflow:Model was constructed with shape (None, 35, 2) for input Tensor("flatten_input:0", 
shape=(None, 35, 2), dtype=float32), but it was called on an input with incompatible shape (None, 2).                                  
Traceback (most recent call last):                                                                                                                                                                                                             
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py", line 60, in 
quick_execute                                                                                                                                 
inputs, attrs, num_outputs)                                                                                                                                                                                                              
TypeError: An op outside of the function building code is being passed                                                                                                                                                                       
a "Graph" tensor. It is possible to have Graph tensors                                                                                                                                                                                       
leak out of the function building context by including a                                                                                                                                                                                     
tf.init_scope in your function building code.                                                                                                                                                                                                
For example, the following function will fail:                                                                                                                                                                                                 
@tf.function                                                                                                                                                                                                                                 
def has_init_scope():                                                                                                                                                                                                                          
my_constant = tf.constant(1.)                                                                                                                                                                                                                
with tf.init_scope():                                                                                                                                                                                                                          
added = my_constant * 2                                                                                                                                                                                                                
The graph tensor has name: dense_1/Identity:0                                                                                                                                                                                                                                                                                                                                                                                                                                             
During handling of the above exception, another exception occurred:                                                                                                                                                                                                                                                                                                                                                                                                                       
Traceback (most recent call last):                                                                                                                                                                                                             
File "addition.py", line 58, in <module>                                                                                                                                                                                                       
eval_data, eval_target, epochs=10                                                                                                                                                                                                          
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py", line 66, in 
_method_wrapper                                                                                                                       
return method(self, *args, **kwargs)                                                                                                                                                                                                       
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py", line 848, in 
fit                                                                                                                                  
tmp_logs = train_function(iterator)                                                                                                                                                                                                        
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\def_function.py", line 580, in 
__call__                                                                                                                                
result = self._call(*args, **kwds)                                                                                                                                                                                                         
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\def_function.py", line 644, in 
_call                                                                                                                                   
return self._stateless_fn(*args, **kwds)                                                                                                                                                                                                   
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 2420, in 
__call__                                                                                                                                   
return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access                                                                                                                                                     
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1665, in 
_filtered_call                                                                                                                             
self.captured_inputs)                                                                                                                                                                                                                      
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1746, in 
_call_flat                                                                                                                                 
ctx, args, cancellation_manager=cancellation_manager))                                                                                                                                                                                     
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 598, in call                                                                                                                                        
ctx=ctx)                                                                                                                                                                                                                                   
File "C:\Users\User\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py", line 74, in 
quick_execute                                                                                                                                 
"tensors, but found {}".format(keras_symbolic_tensors))                                                                                                                                                                                  
tensorflow.python.eager.core._SymbolicException: Inputs to eager execution function cannot be Keras 
symbolic tensors, but found [<tf.Tensor 'dense_1/Identity:0' shape=(None, 70, 1) dtype=float32>]

Any help, tips, and advice is greatly appreciated.

Aidan L.
  • 79
  • 3
  • 14
  • AFAIK this is the roor-cause of error. Why are you adding this `model.add(keras.layers.Lambda( lambda x: tf.expand_dims(model.output, axis=-1)))` – Vishnuvardhan Janapati May 25 '20 at 18:10
  • Oh cause, I received an error awhile ago, saying that there are incompatible shapes. Basically it needs another dimension. Is it causing the problems? – Aidan L. May 25 '20 at 18:16
  • what is the error you get when you remove that lambda layer entirely? I see another warning about `input_shape`? what is the shape of `eval_data` and `eval_target` – Vishnuvardhan Janapati May 25 '20 at 19:12
  • Here you go: (sorry if it's shrunk, I needed all of it on the screen. https://ibb.co/5rqQJbp – Aidan L. May 25 '20 at 19:24
  • what is the shape of eval_data and eval_target. I cannot see entirely but I think I know what is going on. There is a mismatch in your data size and input_shape defined in the first layer – Vishnuvardhan Janapati May 25 '20 at 19:31
  • eval_data = (22, 2), eval_target = (22, ) – Aidan L. May 25 '20 at 19:31
  • I changed the input_shape = (22, 2). But it's still giving me the same error. **ERROR_1:** WARNING:tensorflow:Model was constructed with shape (None, 22, 2) for input Tensor("flatten_input:0", shape=(None, 22, 2), dtype=float32), but it was called on an input with incompatible shape (None, 2) **ERROR_2:** ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 44 but received input with shape [None, 2] – Aidan L. May 25 '20 at 19:33
  • 1
    check the answer below. – Vishnuvardhan Janapati May 25 '20 at 19:41

1 Answers1

1

I think the issue is with the Lambda layer that was taking model.ouput. based on your eval_data and eval_target, I updated the model. So, please check the following model.

model = tf.keras.Sequential()
# model.add(tf.keras.layers.Flatten(input_shape=(2,)))
# model.add(tf.keras.layers.Lambda(lambda x: tf.expand_dims(model.output, axis=-1)))
model.add(tf.keras.layers.Dense(10, activation="tanh",input_shape=(2,)))
model.add(tf.keras.layers.Dense(1, activation="tanh"))
model.summary()

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_4 (Dense)              (None, 10)                30        
_________________________________________________________________
dense_5 (Dense)              (None, 1)                 11        
=================================================================
Total params: 41
Trainable params: 41
Non-trainable params: 0
_________________________________________________________________
Vishnuvardhan Janapati
  • 3,088
  • 1
  • 16
  • 25
  • It seems that the `input_shape` partially cause this error, could you explain more on input_shape and how it works? – Aidan L. May 25 '20 at 19:56
  • 1
    You have 2 features per data which is why you need to set `input_shape=(2,)`, then the TF code automatically adds `None` so that you can change the number of datasets (or examples). As you have 1-dimensional data, you don't need to flatten it. If you have 2D data like MNIST (28,28), then you need `Flatten`. [This tutorial](https://www.tensorflow.org/tutorials/quickstart/beginner) is best place to start. – Vishnuvardhan Janapati May 25 '20 at 20:03
  • I have a few questions. How is my data 1 dimension? And the "2" is that for column or row? Also, what you mean by changing the number of datasets? I don't understand much about this. – Aidan L. May 25 '20 at 20:17
  • 1
    [This article](https://stackoverflow.com/questions/44747343/keras-input-explanation-input-shape-units-batch-size-dim-etc) should give much better idea about shapes. – Vishnuvardhan Janapati May 25 '20 at 21:12
  • 1
    For ex. House prediction data, feature1 is num bedrooms, feature2 is lot size, output is a number (In deep learning lingo it is called as Scalar, with dimension = 0) where as two features will be represented by a vector of two values (dim=1). – Vishnuvardhan Janapati May 25 '20 at 21:16
  • Okay, I see. So just to clarify, what is the dimension of the example above? – Aidan L. May 25 '20 at 21:24
  • 1
    In your example, you have 22 examples of 2 features each. One example is 1 row and 2 columns in your case. – Vishnuvardhan Janapati May 25 '20 at 21:28
  • I see, and that is 1 dimension? – Aidan L. May 25 '20 at 21:32
  • 1
    Yes. It is little difficult and weird lingo depending on your background. There is lot of confusion with dimension, rank, axis, etc. Check for more [here](https://www.tensorflow.org/guide/tensor) – Vishnuvardhan Janapati May 25 '20 at 21:41
  • I think I am starting to get it. So dimensions give a tensor a rank? So [1, 2, 3] <-- Is a rank 1 array. While [ [1, 2, 3] ] <-- is a rank 2 array. And [ [1, 2, 3], [1, 2, 3]] Is a rank 2 array with shape of (2, 2)? – Aidan L. May 25 '20 at 21:48
  • Okay thank you so much! One last question, for the input_shape(), why do should I enter (2, ) and not ( ,2). Since the shape is (35, 2). And also, why should it be (2, ), why leave the other part empty? Sorry, about this, this will be my last question. – Aidan L. May 25 '20 at 22:26
  • The comma is necessary when you have only one dimension. But if you have more than one dimension, you don't need that "," in the end. I think that is how they designed it. Don't know details. The above stackoverflow article has much more details and is worth reading. – Vishnuvardhan Janapati May 25 '20 at 23:55