I had the same problem on Android. Before calling startContactsUpload, try deleting previous contact uploads using deleteAllUploadedContacts. The aforementioned methods are Android APIs but I'm sure there are equivalents for iOS. Also, I think that there's a quota limit per day.
Digits.getInstance().getContactsClient().deleteAllUploadedContacts(new ContactsCallback<Response>() {
@Override
public void success(Result<Response> result) {
Digits.getInstance().getContactsClient().startContactsUpload();
}
@Override
public void failure(TwitterException e) {
e.printStackTrace();
Digits.getInstance().getContactsClient().startContactsUpload();
}
});