How do I implement the function f(x) = { x^2, x>0 and -x^2 , x<0 } for a numpy array 'x'?

Question

I'm trying to change every value in a Numpy array 'x' according to the following function:

f(x) = { x^2 , x >= 0 and 
        -x^2 , x < 0  }

@numpy.vectorize
def squares(x):
    return (x ** 2) if x >= 0 else -(-x ** 2)

The function seemed to have executed properly, but when the result is used for this

tmp = output_errors * output_network * (1.0 - output_network)

later, it shows the following error: RuntimeWarning: overflow encountered in multiply

However, no such error is seen when the following code is run:

@numpy.vectorize
def sigmoid(x):
    return 1 / (1 + numpy.e ** -x)

The entire Code is

import time
import pickle
from scipy.stats import truncnorm
from datetime import datetime

@numpy.vectorize
def activate(x):
    negative_mask = x < 0
    x = x ** 2
    x[negative_mask] *= -1
    return x
    #return (x ** 2) if x > 0 else (-(x ** 2))
    
def truncated_normal(mean=0, sd=1, low=0, upp=10):
    return truncnorm((low - mean) / sd, (upp - mean) / sd, loc=mean, scale=sd)

def create_weight_matrices(no_of_in_nodes, no_of_out_nodes, no_of_hidden_nodes):
    rad = 1 / numpy.sqrt(no_of_in_nodes)
    X = truncated_normal(mean=0, sd=1, low=-rad, upp=rad)
    wih = X.rvs((no_of_hidden_nodes, no_of_in_nodes)) # Weight from Input to Hidden Layer
    
    rad = 1 / numpy.sqrt(no_of_hidden_nodes)
    X = truncated_normal(mean=0, sd=1, low=-rad, upp=rad)
    who = X.rvs((no_of_out_nodes, no_of_hidden_nodes)) # Weight from Hidden to Output Layer
    return wih, who

def train(wih, who, learning_rate, input_vector, target_vector):
    input_vector = numpy.array(input_vector, ndmin=2).T # Input Vector needs to be Transposed
    target_vector = numpy.array(target_vector, ndmin=2).T # Target Vector needs to be Transposed
    
    output_hidden = activate(numpy.dot(wih, input_vector))
    output_network = activate(numpy.dot(who, output_hidden))
    
    output_errors = target_vector - output_network
    tmp = output_errors * output_network * (1.0 - output_network)     
    who += learning_rate  * numpy.dot(tmp, output_hidden.T)

    hidden_errors = numpy.dot(who.T, output_errors)
    tmp = hidden_errors * output_hidden * (1.0 - output_hidden)
    wih += learning_rate * numpy.dot(tmp, input_vector.T)

    return wih, who

def identify(wih, who, input_vector):
    input_vector = numpy.array(input_vector, ndmin=2).T

    output_vector = numpy.dot(wih, input_vector)
    output_vector = activate(output_vector)
    
    output_vector = numpy.dot(who, output_vector)
    output_vector = activate(output_vector)

    return output_vector

image_size = 28 # Specified for the MNIST Dataset
no_of_different_labels = 10 # The Numbers from 0 to 9
image_pixels = image_size * image_size
data_path = "C:/Users/HP/Desktop/Athul/Academics B.Tech/Semester 6/Introduction to Data Communication/Mini Project - Digit Recognition/"

# MNIST Dataset has been pickled to make access faster!!!!
with open(data_path + "pickled_mnist.pkl", "br") as fh:
    data = pickle.load(fh)

train_imgs = data[0]
test_imgs = data[1]
train_labels = data[2]
test_labels = data[3]
train_labels_one_hot = data[4]
test_labels_one_hot = data[5]

no_of_in_nodes = image_pixels
no_of_out_nodes = 10 
no_of_hidden_nodes = 100
learning_rate = 0.2

#Create Weighted Matrix
wih, who = create_weight_matrices(no_of_in_nodes, no_of_out_nodes, no_of_hidden_nodes) #Create a Matrix to store the weights between Nodes

# Train the Neural Network (Assign Values to Weighted Matrix using the whole MNIST Dataset)
print("Initiating Training...")
#print(numpy.unique(train_imgs))
start_time = datetime.now()
for i in range(int(len(train_imgs[:1000]))):
    wih, who = train(wih, who, learning_rate, train_imgs[i], train_labels_one_hot[i])
end_time = datetime.now()
print("Dataset has been Trained!!!\nTime taken = ", end = "")
print(end_time - start_time, end = "\n\n")


with open(data_path + "neural_net.pkl", "bw") as line:
    data = (wih, 
            who)
    pickle.dump(data, line)
print("Neural Net Pickled!!!!")


#Test the Neural Network using MNIST Dataset
count = 0;
max_right = 0
max_wrong = 0
min_right = 1
min_wrong = 1
for i in range(len(test_imgs)):
    result = identify(wih, who, test_imgs[i])
    if(int(test_labels[i][0]) != int(numpy.argmax(result))):
        if(max_wrong < numpy.max(result)):
            max_wrong = numpy.max(result)
        if(min_wrong > numpy.max(result)):
            min_wrong = numpy.max(result)
    else:
        count += 1;
        if(max_right < numpy.max(result)):
            max_right = numpy.max(result)
        if(min_right > numpy.max(result)):
            min_right = numpy.max(result)

accuracy = (count / len(test_imgs) ) * 100.0
print("Accuracy from testing with " + str(len(test_imgs)) + " pictures = " + str(accuracy) + "%")
max_right *= 100
max_wrong *= 100
min_right *= 100
min_wrong *= 100
print("Highest Accuracy of Right matches = " + str(max_right) + "%")
print("Highest Accuracy of Wrong matches = " + str(max_wrong) + "%")
print("Lowest Accuracy of Right matches = " + str(min_right) + "%")
print("Lowest Accuracy of Wrong matches = " + str(min_wrong) + "%")

It trains a Neural Network to recognize digits!!!

`if x >= 0` only works if `x` is a single number, not if it's an array. Python `if` is simpler switch, not any sort of iterator. — hpaulj, Jul 15 '20 at 03:19
***OverflowError: (34, 'Result too large')*** at the Return statement when I changed ```x>=0``` to ```x>0``` — Athul Shibu, Jul 15 '20 at 03:23
Btw., the power operator `**` has higher precedence than the `-` (unary and otherwise), so your `-(-x ** 2)` is the same as `x ** 2`. — Paul Panzer, Jul 15 '20 at 03:54

Evan · Answer 1 · 2020-07-15T03:33:18.893

1

How large (or small, if negative) are the numbers are you trying to square? Overflow means there aren't enough bits to store the result. You could try using a numpy array with a larger dtype. Here's an example of converting an existing array, or you can size it at creation time:

a = a.astype(numpy.float128)

And for the function, vectorizing like you did should work, or:

negative_mask = a < 0
a = a ** 2
a[negative_mask] *= -1

edited Jul 15 '20 at 03:33

answered Jul 15 '20 at 03:28

Evan

111
3

```print(numpy.unique(train_imgs))``` give the result ***[0.01 0.01388235 0.01776471 0.02164706 0.02552941 0.02941176 ... 0.98835294 0.99223529 0.99611765 1. ]*** – Athul Shibu Jul 15 '20 at 03:30
1

What about printing the min() and max() of output_network and output_errors right before the line that crashes? – Evan Jul 15 '20 at 03:37

score 0 · Answer 2 · answered Jul 15 '20 at 03:35

0

You can use this for your function:

def squares(x):
    return (x**2) * np.sign(x)

But your issue seems to be coming from multiplying large numbers. Check out your max/min numbers.

answered Jul 15 '20 at 03:35

Ehsan

12,072
2
20
33

All values are Float values between 0 and 1!!! ***[0.01 0.01388235 0.01776471 0.02164706 0.02552941 0.02941176 ... 0.98835294 0.99223529 0.99611765 1. ]*** – Athul Shibu Jul 15 '20 at 03:37
@AthulShibu What are `output_errors` and `output_network` values? – Ehsan Jul 15 '20 at 03:38
They are parts of the Neural Net (Code available in the question). Everything works when the activation Function is Sigmoid, but I wanted to change it to squares! – Athul Shibu Jul 15 '20 at 03:48
You have to track values of your arrays pre-error to see when overflow happens to find the root of the problem. – Ehsan Jul 15 '20 at 03:58

How do I implement the function f(x) = { x^2, x>0 and -x^2 , x<0 } for a numpy array 'x'?

2 Answers2