3

Couple of questions about Form Recognizer (FR) model management:

Background: I'm using FR Labeling tool to train models and C# Function app to interact with FA service and analyze forms.

  1. Each time a model is trained - a new instance is created. New version does not hold any reference to previous versions and there is no way of selecting a model by name in code. Latest model might be queried using TrainingCompletedOn property but it's not failproof and cannot be used if FR has more than one project. Question: if continuous development is being done and model is constantly improved - is there a way (or best practice) to manage which model has to be targeted.
  2. In connection to 1st Q - since FR always creates a new model - it ends up with a big list of not used models that are still active. and since there is no connection between them - there is no safe way of performing a cleanup. old Models can be removed using API but it's a manual process. Any recommendations on how old model versions can be managed?
  3. Can a model be exported added to version control and deployed to other environments from version control? There is an API endpoint to copy models between FR instances, but I would like to keep it in version control and deploy to environments from there.
  4. In connection to Q3 - What is the recommended practice for managing FR project in DevOps? how can work be versioned and deployed across different environments?

Thank you

Aba
  • 55
  • 6

1 Answers1

0
  1. each model is unique and independent. it's immutable, you need to pick the model with best accuracy based on your test data set.
  2. you can call DELETE api to delete a model.
  3. not such support at this point. as each model can't be changed after it's created, I don't think there is much value in version control for a model.
  4. please see #1, you could use a test data set to measure model performance. If the model does poorly on one test file, you could label that test file and add it to the training set, and train a new (better) model.

-xin (MS Form Recognizer Team)

Xin Zou
  • 312
  • 1
  • 9
  • 1, 2: models are immutable but do (or can) have logical grouping. At least when using a labeling tool, you first create a project (and a storage account container), so all models created in that project should have some kind of logical connection. however, if you train a new model, previous model is 'lost' and can only be managed using API and if you have Model ID. if you have more than one project in one FR instance - there is no way to identify which project model belongs to, which one is the latest/in use, and which ones can be deleted – Aba Aug 24 '20 at 08:16