19

In looking over the new CoreML API, I don't see any way to continue training the model after generating the .mlmodel and bundling it in your app. This makes me think that I won't be able to perform machine learning on my user's content or actions because the model must be entirely trained beforehand.

Is there any way to add training data to my trained model after shipping?

EDIT: I just noticed you could initialize a generated model class from a URL, so perhaps I can post new training data to my server, re-generate the trained model and download it into the app? Seems like it would work, but this completely defeats the privacy aspect of being able to use ML without the user's data leaving the device.

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
Patrick Goley
  • 5,397
  • 22
  • 42
  • 1
    I ask this same question to a Core ML engineer yesterday. They basically said they do not support loading it from a server at this time as it is compiles along with your app. They did hint on maybe supporting it in the future depending on interest. My concern is specifically with security of the model as well as training is an expensive task so it's a valuable resource. I asked about including a partially trained or generic model at compile time and downloading an updated one later and they said it wan't possible. Matthijs' answer below is worth looking into. – aasatt Jun 09 '17 at 22:46
  • @Patrick's original question about on-device training is a good one to cover all by itself. It would be useful if you could make a separate question for the server side training since the answer will cover some different technical areas. – Alex Brown Jun 14 '17 at 16:49

7 Answers7

11

The .mlmodel file is compiled by Xcode into a .mlmodelc structure (which is actually a folder inside your app bundle).

Your app might be able to download a new .mlmodel from a server but I don't think you can run the Core ML compiler from inside your app.

Maybe it is possible for your app to download the compiled .mlmodelc data from a server, copy it into the app's Documents directory, and instantiate the model from that. Try it out. ;-)

(This assumes the App Store does not do any additional processing on the .mlmodelc data before it packages up your app and ships it to the user.)

Matthijs Hollemans
  • 7,706
  • 2
  • 16
  • 23
  • 1
    Use `xcrun coremlcompiler compile /path/to/MyModel.mlmodel /path/to/` to compile your model – Alex Brown Jun 11 '17 at 01:10
  • You can side load mlmodelc files directly into the app if you wish. If that doesn’t work, ask another question about that. – Alex Brown Jun 21 '17 at 05:20
  • 1
    Has anyone confirmed that the compiled model can be downloaded and used on an already live app? – Dylan Jun 27 '17 at 12:43
  • https://developer.apple.com/documentation/coreml/core_ml_api/downloading_and_compiling_a_model_on_the_user_s_device Downloading and Compiling a Model on the User's Device – diegodsp Oct 16 '19 at 13:08
  • We can update on the fly now: https://venturebeat.com/2020/06/24/apples-core-ml-now-lets-app-developers-update-ai-models-on-the-fly/ – Zahid Usman Cheema Jun 03 '21 at 08:24
10

Apple has recently added a new API for on-device model compilation. Now you can download your model and compile it on device

Shehroz
  • 171
  • 3
  • 11
  • Suppose i already trained my model given in app. Now user takes more picture and add it to existing model. Existing model will be updated (merged) with new images. Is it really possible to train an existing model on app programmatically while user uses the app ? – Jamshed Alam Jul 24 '19 at 06:52
3

CoreML 3 now supports on-device model personalization. You can improve your model for each user while keeping its data private.

https://developer.apple.com/machine-learning/core-ml/

Marcos Paulo
  • 317
  • 3
  • 10
2

Core ML supports inference but not training on device.

You can update the model by replacing it with a new one from a server, but that deserves its own question.

Alex Brown
  • 41,819
  • 10
  • 94
  • 108
  • 1
    No. BNNS supports implementation and operation of neural networks for inference, using input data previously derived from training. BNNS does not do training, however. Its purpose is to provide very high performance inference on already trained neural networks. – John Difool Jun 28 '17 at 22:02
2

In order to dynamically update the model (without updating the whole app), you need to use MPS (Metal Performance Shader) directly instead of relying on .mlmodel, which must be bundled with the app.

It means you need to manually build the neural network by writing some Swift code (instead of using coremltools to converts existing models directly), and feed various weights for each layer, which is a little bit of work, but not a rocket science.

This is a good video to watch if you want to know more about MPS.

https://developer.apple.com/videos/play/wwdc2017/608/

Satoshi Nakajima
  • 1,863
  • 18
  • 29
2

Now with iOS11 beta4, you can compile the model, download from server.

(Details)

Stephen Rauch
  • 47,830
  • 31
  • 106
  • 135
michaelxing
  • 83
  • 6
  • this link also can help you , i think. https://developer.apple.com/documentation/coreml/core_ml_api/downloading_and_compiling_a_model_on_the_user_s_device – michaelxing Jan 08 '18 at 09:36
1

As alternative to bundling an mlmodel with application, you can download and then compile models within your CoreML app. For this you just need to download the model definition file onto the user’s device by using, for example, URLSession. And after this you have to compile the model definition by calling a throwing compileModel(at:) type method.

let newCompiledModel = try MLModel.compileModel(at: modelDescriptionURL)

You'll get a new compiled model file with the same name as the model description but its ending will be mlmodelc. Eventually, create a new MLModel instance by passing the compiled model URL to its initialiser.

let model = try MLModel(contentsOf: newCompiledModel)

However, remember that compiling process may be time consuming and it shouldn't be done on main thread.

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220