FastText cannot be opened for loading from bucket

Question

I downloaded fasttext model lid.176.bin. If I run my code locally with model in folder everything works fine. But I need to run it in GC, so, I uploaded model into bucket and changed path to model from local to gs bucket and got an error: ValueError: gs://models/fasttext-model/lid.176.bin cannot be opened for loading! How can I use model from bucket?

path_to_pretrained_model = 'gs://models/fasttext-model/lid.176.bin'
fasttext_model = fasttext.load_model(path_to_pretrained_model)

I wouldn't expect custom-protocol paths (like `gs://models/fasttext-model/lid.176.bin`) to necessarily work with an arbitrary library (like `fasttext`) without special arrangements. (Does Google Cloud purport to ensure that?) Thus, if possible, can you pre-download the model to (what looks like) a local path before loading? — gojomo, Jul 10 '21 at 19:34
Please provide more details, overview. How did you configure your bucket, its region or multiregion, its standard class, its possible to reach it from public? You are trying to run it locally? Please provide steps how you are trying to run in. This is whole error? — PjoterS, Jul 12 '21 at 12:01

score 2 · Answer 1 · answered Jul 13 '21 at 10:24

This function from Google Documentation helps me to solve the problem

from google.cloud import storage


def download_blob(bucket_name, source_blob_name, destination_file_name):
    """Downloads a blob from the bucket."""
    # bucket_name = "your-bucket-name"
    # source_blob_name = "storage-object-name"
    # destination_file_name = "local/path/to/file"

    storage_client = storage.Client()

    bucket = storage_client.bucket(bucket_name)

    # Construct a client side representation of a blob.
    # Note `Bucket.blob` differs from `Bucket.get_blob` as it doesn't retrieve
    # any content from Google Cloud Storage. As we don't need additional data,
    # using `Bucket.blob` is preferred here.
    blob = bucket.blob(source_blob_name)
    blob.download_to_filename(destination_file_name)

    print(
        "Blob {} downloaded to {}.".format(
            source_blob_name, destination_file_name
        )
    )

Hey @m.k I am facing this isse! I am trying to do the exact thing. Loading Fasttext model stored in a bucket. Could you please elaborate how did u fix it? Also I want to use with Dataflow.. — Chaitanya Patil, Aug 12 '21 at 16:24

FastText cannot be opened for loading from bucket

1 Answers1