-1

How can I inherit the ParDo Class (which is in beam.py) to generic Class (which is in generic.py file) or vice versa?

example : beam.py

class rejected_records(beam.DoFn):
    def process(self,element):
"""
Transformation
"""
        return element

generic.py

Class generic():
   def a:

   def b:

   def c:

Thanks in Advance, please provide me solution or links in python

Robert
  • 7,394
  • 40
  • 45
  • 64
umesh km
  • 1
  • 1
    Please provide code that actually compiles. Where is the `ParDo` class you mention? Where is the code where you actually try to inherit? We need a [mcve] to help you, not just some code fragments with an unclear description. See also [ask]. – Robert Jun 03 '22 at 15:21

1 Answers1

0

To define your own DoFn and use it with ParDo in a Beam pipeline, please refer to https://beam.apache.org/documentation/programming-guide/#pardo.

There isn't any special magic to use DoFn with Generic, just do it as normal Python code:

import apache_beam as beam
from typing import Generic, TypeVar
T = TypeVar('T')
class MyDo(beam.DoFn, Generic[T]):
  def process(self, element: T) -> T:
    yield element

p = beam.Pipeline()
int_p = p | 'int create' >> beam.Create([1,2,3])
str_p = p | 'str create' >> beam.Create(['a', 'b', 'c'])
int_do = int_p | 'int do' >> beam.ParDo(MyDo[int]())
str_do = str_p | 'str do' >> beam.ParDo(MyDo[str]())
int_do | 'int print' >> beam.Map(print)
str_do | 'str print' >> beam.Map(print)
p.run()
ningk
  • 1,298
  • 1
  • 7
  • 7