0

Does Amazon SageMaker built-in LightGBM algorithm support distributed training?

I use Databricks for distributed training of LightGBM today. If SageMaker built-in LightGBM supports distributed training, I would consider migrating to SageMaker. It is not clear in the Amazon SageMaker's built-in LightGBM's documentation on whether it supports distributed training.

Thanks very much for any suggestion or clarification on this.

juvchan
  • 6,113
  • 2
  • 22
  • 35

3 Answers3

0

Currently SageMaker LightGBM doesn't support distributed training.

Arun Lokanatha
  • 290
  • 1
  • 4
0

I went through the LightGBM section of SageMaker documentation and there are no references that it supports distributed training. One of the example here uses single instance type. Also looked at lightGBM documentation here . Here are the parameters that you need to specify

tree_learner=your_parallel_algorithm,

num_machines=your_num_machines,

Given I couldnt find any reference of above in SageMaker documentation, I assume its not supported.

Cyril Bandolo
  • 16
  • 1
0

https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-built-in-lightgbm-now-offers-distributed-training-using-dask/

SageMaker LightGBM algorithm offers distributed training using the Dask framework for both tabular classification and regression tasks.

YOSHI
  • 1