5

The documentation says:

If a parameter is created with significant=False, it is ignored as far as the Task signature is concerned. Tasks created with only insignificant parameters differing have the same signature but are not the same instance.

But how does this help with the task-flow in Luigi, because if I want to run two instances of a task, with different arguments, I set significant=True for the parameters, such that Luigi considers them as separate tasks, and such that completion of one task doesn't mark the other task as DONE as well.

But I'm not getting these results by marking the parameters as significant=True . So, what does the significant parameter do, and when should I use it?

JJJ
  • 32,902
  • 20
  • 89
  • 102
Shailesh
  • 2,116
  • 4
  • 28
  • 48

2 Answers2

0

But I'm not getting these results by marking the parameters as significant=True .

Not sure what's the problem in your situation, however,

luigi parameter module

significant (bool) – specify False if the parameter should not be treated as part of the unique identifier for a Task. An insignificant Parameter might also be used to specify a password or other sensitive information that should not be made public via the scheduler. Default: True.

So it's a security practice.

Shihe Zhang
  • 2,641
  • 5
  • 36
  • 57
0

Luigi identifies a task with its parameters to maintain uniqueness. Lets say,

class SampleTask(luigi.Task):
    id = luigi.Parameter(default=uuid.uuid4.__str__())
    file_id = luigi.Parameter(default=uuid.uuid4.__str__(),significant=False)

Luigi while scheduling will name the task as SampleTask_xaxx24_a42sa5rq_a4glkqjw45_asdr i.e (SampleTask_uuid). It is basically the id parameter we have defined. By default significant is True. So all parameters are considered Significant will tell Luigi scheduler whether to consider the second parameter while creating the task or not. This helps the scheduler to understand which will be unique for task name identification when the names clash during dynamic dependency creation.

Pathanjali Tallapragada
  • 101
  • 1
  • 1