2

Because of the current limitation regarding the publish of scikit-learn models on Watson ML service, which does not allow any custom transformer etc (https://datascience.ibm.com/docs/content/analyze-data/ml-scikit-learn.html) in the pipeline, I ended up deploying a pipeline that only contains the SVC classifier, and not the TfidfVectorizer as well.

Which means, I need to "transform" my raw test data with the TfidfVectorizer before invoking the model on Watson ML.

This is working fine as long as I don't try the online deployment approach (which I need, since I want an app to POST a request to my model).

How should I serialise my sparse matrix from the TfidfVectorizer.transform and pass it as a json payload to the WML service ?

Thanks !

Q's
  • 41
  • 2

1 Answers1

2

So actually, I am answering my question ;-)

If you get into that situation where you have to send a sparse matrix to WML, then you can use

<yourmatrix>.todense().tolist()

So, to put it back in context of my initial issue, I can send the result of the transform as such:

valuesList = tfidf_vectorizer.transform(test).todense().tolist()
payload_scoring = {"values": [[valuesList]]}
response_scoring = requests.post(scoringUrl, json=payload_scoring, headers=header)
Q's
  • 41
  • 2