1

I'm trying to deploy a haystack model for Question Answering for my application as a REST API /API. I want to query and get my answers directly and I need to do it soon so I'm finding a way to do it on Algorithmia. Any suggestions, tutorials, examples or any help is appreciated. Thanks!!

For reference, this could be an example model.

avats
  • 437
  • 2
  • 10
  • This sounds like some homework. What have you done? What did you test? It would be helpful for others to diagnose the problem. – Melon Oct 12 '21 at 14:16
  • No, it was for a hackathon I was participating in, and even after a lot of research I was unable to deploy it in time. But, now I am trying to deploy it using heroku as a REST api as haystack provides a built in rest api example (build using fastapi). I am still looking for a solution that uses algorithmia to deploy this. – avats Oct 13 '21 at 16:55

1 Answers1

1

Not sure about Alorithmia, but here's a simple option to deploy a Haystack service incl. a REST API on any standard machine (e.g. AWS EC2 instance):

# Clone haystack repo
git clone https://github.com/deepset-ai/haystack.git
cd haystack
# Start (demo) containers
docker-compose pull
docker-compose up
# Run a query
curl -X 'POST' \
  'http://127.0.0.1:8000/query' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "Who is the father of Arya Stark?",
  "params": {}
  }'

This basically spins up:

  1. Haystack REST API using this docker image
  2. Elasticsearch with some demo data (see comment in the docker-compose.yaml for how to replace this with an empty instance for your own data)
  3. A simple streamlit-based UI (you can easily remove this from the docker-compose if you don't need it)

If you want to customize the pipeline being deployed in the API (e.g. change a model):

  • Edit the pipelines.yaml in your cloned repo (in haystack/rest_api/pipeline/)
  • Mount this folder as a volume into the container by uncommenting this part in the docker-compose.yaml

If you want to deploy on a GPU machine, just execute instead:

docker-compose -f docker-compose-gpu.yml pull
docker-compose -f docker-compose-gpu.yml up

For more details, see the official documentation of the REST API here.

Malte
  • 895
  • 2
  • 8
  • 16