I have implemented stratified cross validation for multiclass imbalanced dataset. Im unable to calculate the average of each performance metric such as precision, recall etc.
skf = StratifiedKFold(n_splits=10)
lst_accu_stratified = []
lst_pre_stratified = []
for train_index, test_index in skf.split(X, y):
print("TRAIN:", train_index, "TEST:", test_index)
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y.iloc[train_index], y.iloc[test_index]
dt.fit(X_train, y_train)
# score = dt.score(X_test,y_test)
lst_accu_stratified.append(dt.score(X_test, y_test))
lst_pre_stratified.append(cross_val_score(dt, X_test, y_test, scoring='precision_weighted'))
# Print the output.
print('List of possible accuracy:', lst_accu_stratified)
print('\nOverall Accuracy:',mean(lst_accu_stratified)*100, '%')
print('\nStandard Deviation is:', stdev(lst_accu_stratified))
#print('For Fold {} the accuracy is {}'.format(str(fold_no),score))
# Print the output.
print('List of possible pre:', lst_pre_stratified)
print('\nOverall pre:',mean(lst_accu_stratified)*100, '%')
print('\nStandard Deviation pre is:', stdev(lst_pre_stratified))
the error it is giving me is this, how can I calculate the weighted average of precision, recall, f1-score etc.
List of possible accuracy: [0.9835704251505708, 0.9982440134988939, 0.9977959341848186, 0.998433740776025, 0.9956645298800277, 0.9979089632009818, 0.9963467259802279, 0.9915873778373425, 0.998042168066752, 0.9966494834956786]
Maximum Accuracy That can be obtained from this model is: 99.8433740776025 %
Minimum Accuracy: 98.35704251505707 %
Overall Accuracy: 99.54243362071318 %
Standard Deviation is: 0.00463447140029694
List of possible pre: [array([0.97498825, 0.99018204, 0.99331666, 0.99531447, 0.99747649]), array([0.99927386, 0.99946234, 0.99961679, 0.99508796, 0.99812511]), array([0.99728536, 0.99963718, 0.99672916, 0.99722338, 0.99667466]), array([0.99927476, 0.99969985, 0.99953702, 0.99964024, 0.99130982]), array([0.99740928, 0.99812025, 0.99882627, 0.99528916, 0.99861694]), array([0.99963661, 0.99965769, 0.99830287, 0.99917217, 0.99826977]), array([0.99796059, 0.99895001, 0.99828052, 0.99729752, 0.99235937]), array([0.99247648, 0.99593567, 0.99934078, 0.99971908, 0.9985657 ]), array([0.99834264, 0.99890637, 0.99885082, 0.99938173, 0.99332253]), array([0.99848042, 0.99981842, 0.99947901, 0.99730492, 0.99834253])]
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-132-03190719070c> in <module>
28 # Print the output.
29 print('List of possible pre:', lst_pre_stratified)
---> 30 print('\nMaximum Accuracy That can be obtained from this model is:', max(lst_pre_stratified)*100, '%')
31 print('\nMinimum Accuracy:', min(lst_pre_stratified)*100, '%')
32 print('\nOverall pre:',mean(lst_accu_stratified)*100, '%')
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Can anyone help how to calculate precion, AUC etc.??
