0

Disclaimer: I am very new to Neural Network and Tensorflow.

I am trying to create a QA application where user asks a question and the application gives the answer. Most of the traditional methods I tried did not work or is not accurate enough or requires manual intervention. I was researching about unsupervised QA application, that is when I came across BERT.

BERT as google claims is state of the art neural network model and achieved highest score in leader board for Squad 2.0. I wish to use this model for my application and test it's performance.

I have created a Windows 2012 Datacenter edition Virtual Machine in Compute Engine. I have created Cloud TPU using ctpu.

I have the BERT large uncased model in Cloud Storage.

How do I train the BERT large uncased model with SQUAD 2.0?

Please feel free to correct me if I am wrong, I have the understanding that Cloud TPU is just a device like CPU or GPU. However if you read this, they are explaining like Cloud TPU is a virtual machine ("On Cloud TPU you can run with BERT-Large as...").

Where do I run run_squad.py as mentioned in here?

python run_squad.py \
  --vocab_file=$BERT_LARGE_DIR/vocab.txt \
  --bert_config_file=$BERT_LARGE_DIR/bert_config.json \
  --init_checkpoint=$BERT_LARGE_DIR/bert_model.ckpt \
  --do_train=True \
  --train_file=$SQUAD_DIR/train-v2.0.json \
  --do_predict=True \
  --predict_file=$SQUAD_DIR/dev-v2.0.json \
  --train_batch_size=24 \
  --learning_rate=3e-5 \
  --num_train_epochs=2.0 \
  --max_seq_length=384 \
  --doc_stride=128 \
  --output_dir=gs://some_bucket/squad_large/ \
  --use_tpu=True \
  --tpu_name=$TPU_NAME \
  --version_2_with_negative=True

How to access the the storage bucket files from Virtual Machine for this argument vocab_file?

Is the external IP address the value for $TPU_NAME environment variable?

Community
  • 1
  • 1
Jeeva Bharathi
  • 99
  • 2
  • 12

1 Answers1

1

So TPUs currently only read from GCS. The model that you've downloaded should be uploaded to another GCS bucket of your own creation. That's how the TPU will access vocab_file and other files.

Alex Ilchenko
  • 322
  • 1
  • 3
  • 1
    How do I say in VM that this gs://example is the bucket I am trying to access? I tried to do that by creating a environment variable with value as 'gs://example'. It threw an error. – Jeeva Bharathi Jun 18 '19 at 05:40
  • You should be trying to access the buckets that are defined by the environment variables. If you go here, https://github.com/google-research/bert, you should see that you are required to do `export BERT_BASE_DIR=gs://bert_models/2018_10_18/uncased_L-12_H-768_A-12` for example, if you are trying to use the base model. – Alex Ilchenko Jun 21 '19 at 18:37