Found input variables with inconsistent numbers of samples: [24, 25]

Question

I need assistance reshaping my input to match my output. I believe my issue is with my target variable. I am getting the error as stated in the title. I have tried .reshape and .flatten(). Please help, and thanks in advance

NEnews_train = []
for line in open('/Users/db/Desktop/NE1.txt', 'r'):
    NEnews_train.append(line.strip())



REPLACE_NO_SPACE = re.compile("[.;:!\'?,\"()\[\]]")
REPLACE_WITH_SPACE = re.compile("(<br\s*/><br\s*/>)|(\-)|(\/)")

def preprocess_reviews(reviews):
    reviews = [REPLACE_NO_SPACE.sub("", line.lower()) for line in reviews]
    reviews = [REPLACE_WITH_SPACE.sub(" ", line) for line in reviews]


    return reviews

NE_train_clean = preprocess_reviews(NEnews_train)

from nltk.corpus import stopwords

english_stop_words = stopwords.words('english')
def remove_stop_words(corpus):
    removed_stop_words = []
    for review in corpus:
        removed_stop_words.append(
            ' '.join([word for word in review.split() 
                      if word not in english_stop_words])
        )
    return removed_stop_words

no_stop_words = remove_stop_words(NE_train_clean)



ngram_vectorizer = CountVectorizer(binary=True, ngram_range=(1, 2))
ngram_vectorizer.fit(no_stop_words)
X = ngram_vectorizer.transform(no_stop_words)
X_test = ngram_vectorizer.transform(no_stop_words)

target = [1 if i < 12 else 0 for i in range(25)]

X_train, X_val, y_train, y_val = train_test_split(
    X, target, train_size = 0.75
)

Here's the error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-14-281ec07b46bb> in <module>
      2 
      3 X_train, X_val, y_train, y_val = train_test_split(
----> 4     X, target, train_size = 0.75
      5 )

~/opt/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py in train_test_split(*arrays, **options)
   2094         raise TypeError("Invalid parameters passed: %s" % str(options))
   2095 
-> 2096     arrays = indexable(*arrays)
   2097 
   2098     n_samples = _num_samples(arrays[0])

~/opt/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in indexable(*iterables)
    228         else:
    229             result.append(np.array(X))
--> 230     check_consistent_length(*result)
    231     return result
    232 

~/opt/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_consistent_length(*arrays)
    203     if len(uniques) > 1:
    204         raise ValueError("Found input variables with inconsistent numbers of"
--> 205                          " samples: %r" % [int(l) for l in lengths])
    206 
    207 

ValueError: Found input variables with inconsistent numbers of samples: [24, 25]

I saw people have similar errors but their code is a bit different from mine, so I got a bit confused on trying to solve

What is the shape of `X` here? It looks like `X` and `target` is not of same length. `train_test_split` requires `X.shape[0] == target.shape[0]` to be `True` — Kousik, Nov 18 '19 at 17:46

Kousik · Answer 1 · 2019-11-18T18:07:43.050

In your X list the total number of item is 24. But in your target array you are taking 25 values ( because range(25) returns an array of [0, 1, 2, ..., 24] total '25' values excluding 25) that's why train_test_split is giving the above error because train_test_split requres X.shape[0] == target.shape[0] to be true.

Solution:

target = [1 if i < 12 else 0 for i in range(25)]

Change above code with the below code if you want to start from 1 to `25(included)``

target = [1 if i < 12 else 0 for i in range(1,26)]

Or if you want to start from 0 to 23(included) then

target = [1 if i < 12 else 0 for i in range(24)]

Found input variables with inconsistent numbers of samples: [24, 25]

1 Answers1