2

I am working on Object Detection and I have two record files. Train.tfrecord(1.6GB) and Test.tfrecord(65MB) file. How do I load the training file in Saturn cloud, as I want to speed up the training time using Dask in Saturn Cloud?

uNIKx
  • 123
  • 2
  • 13
  • Maybe it's best to upload it to S3 (Saturn runs on AWS, right?) and then pass the S3 link to workers. – SultanOrazbayev Mar 17 '21 at 10:25
  • I am newbie. I have signedup on Saturn and trying to upload my 1.6GB file to run it. It is taking very long time to upload and gets stuck in between. I have also make a public bucket and upload both train and test tfrecords. How do I connect with my notebook? Any pointers, link or example will be of great help. Thanks – uNIKx Mar 17 '21 at 10:31

1 Answers1

1

As @SultanOrazbayev mentioned, since Saturn runs on AWS, the best way is to put your data in an S3 bucket. Then you can access it using whichever library you prefer. Normally we recommend using s3fs.

Note: I work at Saturn Cloud.

jsignell
  • 3,072
  • 1
  • 22
  • 23