Cannot deploy a trained model via custom prediction routine. Fails with model load error

Question

I am trying to deploy a custom prediction routine, that categories text. I trained a model and provided the .pkl in the cloud storage model directory. Model version creation fails with:

ERROR: (gcloud.beta.ai-platform.versions.create) Bad model detected with error:  "Failed to load model: User-provided package text_categorizer-0.2.tar.gz failed to install: Command '
['python-default', '-m', 'pip', 'install', '--target=/tmp/custom_lib', '--no-cache-dir', '-b', '/tmp/pip_builds', '/tmp/custom_code/text_categorizer-0.2.tar.gz']' returned non-zero e
xit status 1 (Error code: 0)"

Also are there any logs I could check to know more on why model could not be loaded? The Predictor classes are all available in .tar.gz file uploaded on the cloud storage. Ensured the bucket is in the same region as the model been created. The model files are in a dir called model while the tar.gz file is colocated under the main dir called textcateg. The tar.gz file contains the predictor python class.

Command to create the version:

gcloud beta ai-platform versions create ver3 --model text_categorizer --runtime-version 1.13 --python-version 3.5 --origin gs://mdmceml-bucket/textcateg/model/ --package-uris gs://mdmceml-bucket/textcateg/text_categorizer-0.2.tar.gz --prediction-class predictor.GCPPredictor

Relevant code snippet from GCPPredictor::

    def __init__(self, le, clf):
        """Stores artifacts for prediction. Only initialized via `from_path`.
        """
        self._le = le
        self._clf = clf

    @classmethod
    def from_path(cls, model_dir):
        """Creates an instance of Predictor using the given path.

        Loading of the predictor should be done in this method.

        Args:
            model_dir: The local directory that contains the exported model
                file along with any additional files uploaded when creating the
                version resource.

        Returns:
            An instance implementing this Predictor class.
        """

        model_path = os.path.join(model_dir, 'category_classifier.pkl')
        with open(model_path, 'rb') as f:
            model = pickle.load(f)

        le_path = os.path.join(model_dir, 'labelencoder.pkl')
        with open(le_path, 'rb') as f1:
            le = pickle.load(f1)

        return cls(le, model)

Model version creation fails with:

ERROR: (gcloud.beta.ai-platform.versions.create) Bad model detected with error:  "Failed to load model: User-provided package text_categorizer-0.2.tar.gz failed to install: Command '
['python-default', '-m', 'pip', 'install', '--target=/tmp/custom_lib', '--no-cache-dir', '-b', '/tmp/pip_builds', '/tmp/custom_code/text_categorizer-0.2.tar.gz']' returned non-zero e
xit status 1 (Error code: 0)"