0

This is my first time participating in a kaggle competition and I'm having trouble submitting my result table. I made my model using gbm and made a prediction table like below. the submission file has 2 column named 'fullVisitorId' and 'PredictedLogRevenue') as any other kaggle competition cases.

pred_oob = predict(object = model_gbm, newdata = te_df, type = 'response')

mysub = data.frame(fullVisitorId = test$fullVisitorId, Pred = pred_oob)
mysub = mysub %>%
  group_by(fullVisitorId) %>%
  summarise(Predicted = sum(Pred))

submission = read.csv('sample_submission.csv')
mysub = submission %>%
  left_join(mysub, by = 'fullVisitorId')

mysub$PredictedLogRevenue = NULL
names(mysub) = names(submission)

But when I try to submit the file, I got the 'fail' message saying ...

ERROR: The value '8.893887e+17' in the key column 'fullVisitorId' has already been defined (Line 549026, Column 1)
ERROR: The value '8.895317e+18' in the key column 'fullVisitorId' has already been defined (Line 549126, Column 1)
ERROR: The value '8.895317e+18' in the key column 'fullVisitorId' has already been defined (Line 549127, Column 1)

Not just 3 lines, but 8 more lines like this. I have no idea what I did wrong. I also checked other kernels but couldn't find the answer. Please...help!!

Jean
  • 75
  • 2
  • 11

2 Answers2

2

This issue was because fullVisitorId was numeric instead of character, so It dropped all the leading zeros. Therefore, using read.csv() with colClases argument or fread() can make it work. I left this just because there could be someone else who are having the similar trouble like me

Jean
  • 75
  • 2
  • 11
0

For creating submission dataframe, the easiest way is this

subm_df = pd.read_csv('../input/sample_submission.csv')
subm_df['PredictedLogRevenue'] = <your prediction array>
subm_df.to_csv('Subm_1.csv', index=False)

Noe this is assuming your sample_submission.csv has all fullVisitorId, which it usually does in Kaggle. Following this, I have never faced any issues.

Ankit Paliwal
  • 156
  • 4
  • Thanks Ankit. but this case the testset has 804684 fullVisitorIds and the submission file has 617242 fullVisitorIds. I used group_by() due to this duplication and when it is grouped, the number of row becomes 617242. So I just can't put my prediction array into submission table. – Jean Sep 26 '18 at 11:50
  • In that case you can just create create `PredictedLogRevenue` column in your test dataframe, ad then remove other feature columns, just keeping `fullVisitorId` and `PredictedLogRevenue`, and then submit that. – Ankit Paliwal Sep 26 '18 at 12:04
  • I think you should put this query in the Discussion thread of the respective Kaggle competition or Dataset.If someone already faced a similar issue like this, he/she would be able to help you better. – Ankit Paliwal Sep 26 '18 at 16:36
  • Ok I got it. Thanks Ankit:) – Jean Sep 27 '18 at 05:24