Stack Overflow
Stack Overflow

Questions tagged [serving]

93 questions
4
votes
2 answers

Tensorflow : serving model return always the same prediction

I need your help I'm a little bit stuck right now. I retrain a classification tensorflow model that gives quite nice results. Now I want to serve it through tensorflow serving. I managed to serve it but when i'm using it, it always gives me same…
Machin
  • 41
  • 5
4
votes
0 answers

How to inference Tensorflow model with input queue pipeline?

I am struggling with my Tensorflow model. I have trained it using tf.PaddingFIFOQueue and then I mainly used this tutorial: https://blog.metaflow.fr/tensorflow-how-to-freeze-a-model-and-serve-it-with-a-python-api-d4f3596b3adc#.dykqbzqek, for…
adam13
  • 91
  • 4
4
votes
2 answers

Nginx: serving static files by URL path

Is there any way of serving static files by only some URL path? For example, next URL pattern http://host/static/*.png has /static/ substring (path), and Nginx will serve any statics from there. In the web server documentation I found an…
akrisanov
  • 3,212
  • 6
  • 33
  • 56
3
votes
1 answer

bazel build tensorflow serving using with local downloaded tensorflow

the tensorflow serving build denpend on large tensorflow; but i already build tensorflow successfully. so i want to use it. I do these things: I change the tensorflow serving WORKSPACE(org:…
wingo zhang
  • 31
  • 4
3
votes
2 answers

How do I export a graph to Tensorflow Serving so that the input is b64?

I have a Keras graph with a float32 tensor of shape (?, 224, 224, 3) that I want to export to Tensorflow Serving, in order to make predictions with RESTful. Problem is that I cannot input tensors, but encoded b64 strings, as that is a limitation of…
user3337758
  • 115
  • 1
  • 12
3
votes
0 answers

how to transform checkpoint to savemodel used in tensorflow serving

I trained a model with char-rnn-tensorflow (https://github.com/sherjilozair/char-rnn-tensorflow). The mode is saved into checkpoint. Now I want to serve the model with tensorflow serving. Googled lots of tutorials about this, only found this meet my…
Tony Wang
  • 971
  • 4
  • 16
  • 33
3
votes
2 answers

Google Cloud Storage and temporary serving urls

im wondering if its possible to create a temporary serving url for files on Google Cloud Storage. i have a GAE app and i would like to serve bigger video files over google storage for develpers without 'straming' the files over my appengine app like…
aschmid00
  • 7,038
  • 2
  • 47
  • 66
3
votes
1 answer

How to serve static files in nginx conditionally based on proxy response?

I have a folder /static/uploads/limited and I need to config nginx to serve some files to specific users. I also have an API /api/auth/canIDownloadThis which responses a json like { "result:"true" } How can I make nginx to check the response of…
Farshid Ashouri
  • 16,143
  • 7
  • 52
  • 66
2
votes
1 answer

Unrecognized content type parameters: format when serving model on databricks experiement

I got this Error when serving a model into databricks using MLflow, Unrecognized content type parameters: format. IMPORTANT: The MLflow Model scoring protocol has changed in MLflow version 2.0. If you are seeing this error, you are likely using an…
Sara
  • 353
  • 1
  • 3
  • 13
2
votes
1 answer

Azure Databricks model serving mlflow error version

Hello or good evening everyone ! I recently wanted to use azure's registered models. I attached my experiment to my model and launched the serving, so far so good ! The problem comes from the requests we are trying to make to this…
Akael
  • 61
  • 6
2
votes
0 answers

Custom MLFlow scoring_server for model serving

I would like to know if MLflow currently does support any kind of customization of it's scoring_serving that would allow the ability to register new endpoints to the published Rest API. By default the scoring server provides /ping and /invocations…
jarey
  • 323
  • 2
  • 8
2
votes
1 answer

Tensorflow model quantization best strategy

I'm perplexed by the Tensorflow post-training quantization process. The official site refers to Tensorflow Lite Quantization. Unfortunately, this doesn't work in my case, that is, TFLiteConverter returns errors for my Mask RCNN model: Some of the…
Mariano Italiano
  • 301
  • 2
  • 8
2
votes
1 answer

XGBClassifier: Bad predictions after training, saving and loading a model

Below is how I trained an XGBClassifier and saved it: import pickle from xgboost import XGBClassifier # train model = XGBClassifier() model.fit(X, y) # export pickle.dump(model, open('model.pickle', 'wb')) This is how I loaded the model and made…
mohaseeb
  • 389
  • 4
  • 8
2
votes
1 answer

tensorflow/serving with top n logits to return

I'm currently dealing with the challenge to serve my tensorflow models in a scalable way. As far as I know the recommended solution is to use the standard TensorFlow ModelServer. Common requirements are pretty well handled by this - but I want more.…
Tom
  • 33
  • 5
2
votes
1 answer

SageMaker TensorFlow serving stack comparisons

SageMaker seems to give examples of using two different serving stacks for serving custom docker images: NGINX + Gunicorn + Flask NGINX + TensorFlow Serving Could someone explain to me at a very high level (I have very little knowledge of network…
Austin
  • 6,921
  • 12
  • 73
  • 138
1
2
3 4 5 6 7