0

we are trying to run trace route from airflow (in gcp) to get the output

traceroute = subprocess.Popen(["traceroute", '-w', '100','10.10.10.00'],stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

for line in iter(traceroute.stdout.readline,""):
    print(line)

but getting below error :

FileNotFoundError: [Errno 2] No such file or directory: 'traceroute'

is there any way to run traceroute from gcp airflow ?

Arya
  • 528
  • 6
  • 26
  • 2
    Traceroute is not normally installed with Linux. It is an optional package. Since Composer is a managed service, I am not aware of a method to install Linux packages. – John Hanley May 17 '22 at 07:37

1 Answers1

3

You can use native Python to get the same result as traceroute. You can use pip package mtrpacket and then use the following code. To install a pip package use the PYPI PACKAGES tab from the composer UI.

This code will run without requiring root permissions.

import asyncio
import mtrpacket

async def trace():
    async with mtrpacket.MtrPacket() as mtr:
        for ttl in range(1, 256):
            result = await mtr.probe('172.217.160.206', ttl=ttl)
            print(result)

            if result.success:
                break

asyncio.get_event_loop().run_until_complete(trace())
Kunal Deo
  • 2,248
  • 2
  • 18
  • 27
  • Thanks a lot Kunal but i am getting below error Failed to install PyPI packages. google-cloud-bigquery-storage 2.13.1 has requirement google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5, but you have google-api-core 1.31.3 – Arya May 23 '22 at 03:00
  • Any suggestion pls – Arya May 23 '22 at 03:01
  • @Arya that is a completely different issue, which has nothing to do with Kunal's suggestion. Take a look at your `requirements.txt` file. You've likely locked `google-api-core` to a specific version. – Edo Akse May 23 '22 at 17:12
  • Try it from the composer ui tab as mentioned. – Kunal Deo May 24 '22 at 10:31
  • @KunalDeo . If we are getting error : "failure to communicate with subprocess "mtr-packet" " -- How can we fix this ? – Arya Jun 02 '22 at 14:32