1

In previous versions of Hadoop where MR is used we have a set parameter for ignoring certain percentage of vertex failures.In the new versions where MR is deprecated, is there any equivalent parameter for mapreduce.map.failures.maxpercent in TEZ?

Pruthvi
  • 65
  • 1
  • 1
  • 8

1 Answers1

1

I this Jira new configuration property tez.vertex.failures.maxpercent was added:

Float value. Specifies the allowable percentage in the range 0.0-100.0f of task failures per vertex that will allow the vertex to succeed with failures. Default is 0.0f

See also TezConfiguration.java

leftjoin
  • 36,950
  • 8
  • 57
  • 116
  • I tried to use the parameter but its throwing an error out "Error while processing statement: Cannot modify tez.vertex.failures.maxpercent at runtime. It is not in list of params that are allowed to be modified at runtime". Please let me know if I had to set any other properties for this to work – Pruthvi Feb 18 '20 at 18:06
  • 1
    @Pruthvi Maybe whitelist will help: https://community.cloudera.com/t5/Support-Questions/It-is-not-in-list-of-params-that-are-allowed-to-be-modified/td-p/182820 – leftjoin Feb 18 '20 at 19:21