Stack Overflow
Stack Overflow

Questions tagged [cross-validation]

Cross-Validation is a method of evaluating and comparing predictive systems in statistics and machine learning.

Cross-Validation is a statistical method of evaluating and comparing learning algorithms by dividing data into two segments: one used to learn or train a model and the other used to validate the model.

In typical cross-validation, the training and validation sets must cross-over in successive rounds such that each data point has a chance of being validated against. The basic form of cross-validation is k-fold cross-validation.

Other forms of cross-validation are special cases of k-fold cross-validation or involve repeated rounds of k-fold cross-validation.

2604 questions
1
vote
3 answers

How do I improve my Neural Network output?

I have a data set with 150 rows, 45 features and 40 outputs. I can well overfit the data but I cannot obtain acceptable results for my cross validation set. With 25 hidden layers and quite large number of iterations, I was able to get ~94% accuracy…
cmelan
  • 247
  • 5
  • 13
1
vote
1 answer

How to create learning curve from cross-validated data?

I have an algorithm which uses 10 fold cross validation. Within the training set, I use one of the folds for validation of the training model before using the learned model on the fold held aside for testing. I would like to create a learning curve…
Nick
  • 1,743
  • 6
  • 23
  • 38
1
vote
1 answer

How to perform multi-class cross-validation for LIBSVM in MatLab

I want to use LIBSVM in MatLab to do some multi-class classification. I have read that LIBSVM use One vs. One by default when provided with multiple labels, and I am fine with it. My question is about the parameter search and the model validation.…
Mathieu D.
  • 13
  • 3
1
vote
0 answers

How cross validation is used with LibSVM's java library?

I am trying to perform k-fold cross validation with LibSVM's java library. how far i have understood the cross validation process, the svm_cross_validation() method should give me the optimised values of parameters. Where svm_cross_validation()…
Himanshu Mehra
  • 115
  • 9
1
vote
2 answers

Scikit-learn - Can you run RandomizedSearchCV without cross validation?

I was wondering if you can run RandomizedSearchCV without cross validation (just using a simple train/test split? I want to do this to be able to ballpark what parameters will be useful for more fine-grained tuning where I would use standard cross…
anthonybell
  • 5,790
  • 7
  • 42
  • 60
1
vote
1 answer

Cross-Validation with libsvm to find best parameters

In order to find the best parameters to be used with libsvm I used the code below. Instead of './heart_scale' I had a file containing positive and negative examples each with a hog vector in libsvm format. I had 1000 positive examples and 4000…
Sambas23
  • 683
  • 1
  • 13
  • 27
1
vote
1 answer

Cross-validation in Pybrain

I'm trying to figure out the right way to do 5-fold cross-validation in pybrain. I went through their documentation, but that didn't help. I found the following two versions of code online: Found this one in a question here. net =…
Shruti
  • 159
  • 1
  • 15
1
vote
0 answers

Cross-validating a clmm/clmm2 model

Does anyone have any suggestions to cross-validate a clmm/clmm2 (ordinal package) model in R? I am a bit of a stats novice so apologies... I assume it could be easy to do with the cvTools package in R by writing a cost function. Here is some dummy…
cndmoose
  • 21
  • 4
1
vote
0 answers

Cross validation in logistic regression

I want to perform cross validation in logistic regression using arr as input from load_data function. I have code outline here. The function runs but does not give output. import pandas as pd import numpy as np from sklearn.linear_model.logistic…
Alph
  • 391
  • 2
  • 7
  • 18
1
vote
1 answer

Custom scorer in SciKit-Learn - allow grid search optimisation for a particular class

I would like to create a custom scorer in SciKit-Learn that I can pass to GridSearchCV, which evaluates model performance based upon the accuracy of predictions for a particular class. Suppose that my training data consists of data-points belonging…
Dman2
  • 700
  • 4
  • 10
1
vote
1 answer

Supervised Learning validation Set - ANN

I have implemented a Neural Network in Processing using Supervised Learning method. What I'm actually doing is training some circles to move on their target position. My code works perfectly however, I found that many people split their code into 3…
Apollon1954
  • 1,388
  • 4
  • 16
  • 33
1
vote
1 answer

Cross Validation WEKA random

WEKA Cross Validation: Classifier cls = new J48(); Evaluation eval = new Evaluation(data); Random rand = new Random(1); // using seed = 1 int folds = 10; eval.crossValidateModel(cls, data, folds, rand); …
vubo
  • 45
  • 5
1
vote
1 answer

Use k-fold cross-validation with CRFSuite and save it in a model file

I'm new with the CRFSuite library but I know how to train a model and save it in a file thanks to the "-m" option. However, I try to master de k-fold cross-validation but the "-m" option seem to not working to save de process in a file. The command…
simonboutin1
  • 11
  • 2
1
vote
1 answer

k-fold cross-validation to find misclassification rate

I am trying to write a function which takes a binary response variable y and a single explanatory variable x, runs 10-fold cross validation and returns the proportion of the response variables y that are incorrectly classified. I then want to run…
Rory78
  • 15
  • 1
  • 4
1
vote
2 answers

Cross Validation and perfcurv in Matlab

I am trying to use perfcurv in a cross validation code. However at some point all the members of the test dataset are of the same class (0). My problem is a binary classification problem. Therefore the following error occurs: Error using perfcurve…
obelix
  • 880
  • 2
  • 16
  • 43
1 2 3
…
99 100