0

I'm facing BrokenPipeError when I'm trying to run sentiment analysis with hugging face. It's returning [Error No] 32 Broken Pipe.

Link with total code 'https://colab.research.google.com/drive/1wBXKa-gkbSPPk-o7XdwixcGk7gSHRMas?usp=sharing'

The code is

def create_data_loader(df, tokenizer, max_len, batch_size):
  ds = GPReviewDataset(
    reviews=df.content.to_numpy(),
    targets=df.sentiment.to_numpy(),
    tokenizer=tokenizer,
    max_len=max_len
  )
  return DataLoader(
    ds,
    batch_size=batch_size,
    num_workers=4
  )

Followed by below code

BATCH_SIZE = 16
train_data_loader = create_data_loader(df_train, tokenizer, MAX_LEN, BATCH_SIZE)
val_data_loader = create_data_loader(df_val, tokenizer, MAX_LEN, BATCH_SIZE)
test_data_loader = create_data_loader(df_test, tokenizer, MAX_LEN, BATCH_SIZE)

Followed by

data = next(iter(train_data_loader))
data.keys()

I'm facing error with this 'data = next(iter(train_data_loader))' code

Error is BrokenPipeError: [Errno 32] Broken pipe

Nithin Reddy
  • 580
  • 2
  • 8
  • 18
  • Can you please edit your question and post the full error stacktrace? The class definition of `GPReviewDataset` and `DataLoader` are also helpful to help you. – cronoik Mar 05 '21 at 11:31
  • I've attached colab notebook link. Can you please look into that? It would be really helpful – Nithin Reddy Mar 05 '21 at 13:06
  • I can actually not reproduce this issue. Please include a minimal reproducible example that is causing this error. – cronoik Mar 14 '21 at 00:08
  • Resolved the error by changing num_workers from 4 to 0. If num_workers = 0, then only it'll run on local pc. – Nithin Reddy Mar 15 '21 at 05:41

0 Answers0