-3

I'm trying to find a way to de-reference goldenClusterID to use it in an AWS CLI command to terminate my cluster. This program is to compensate for dynamic Job-Flow Numbers generated each day so normal Data Pipeline shutdown via schedule is applicable. I can os.system("less goldenClusterID") all day and it gives me the right answer. However, it won't give up the goodies with a straight de-ref. Suggestions?

from __future__ import print_function

import json
import urllib
import boto3
import commands
import os
import re
import datetime
import awscli

foundCluster = ""
rawClusterNum = ""
mainClusterNum = ""
goldenClusterID = ""

#   Next, we populate the list file with clusters currently active
os.system("aws emr list-clusters --active >> foundCluster")
#   We search for a specific Cluster Name
os.system("fgrep 'AnAWSEMRCluster' foundCluster")
os.system("grep -B 1 DrMikesEMRCluster foundCluster >> rawClusterNum")
#   Look for the specific Cluster ID in context with it's Cluster Name
os.system("fgrep 'j-' rawClusterNum >> mainClusterNum")
#   Regex the Cluster ID from the line
os.system("grep -o '\j-[0-9a-zA-Z]*' mainClusterNum >> goldenClusterID")
#   Read the Cluster ID from the file and run AWS Terminate on it
os.system("aws emr describe-cluster --cluster-id %s" % goldenClusterID")
os.system("aws emr terminate-clusters --cluster-ids goldenClusterID")
os.system("rm *")
Mike Addison
  • 23
  • 6
  • 2
    Seems weird to use Python for something that looks so much like a Bash script. Otherwise `open("goldenClusterID").read()`. The `goldenClusterID` you're looking for is a file. – F.X. Jul 19 '16 at 21:23
  • @F.X. Normally, I'd just use BASH and set a cron to run it however, this had to be included in a AWS Lambda with it's scheduler so, my choices from AWS, couldn't inlcude straight BASH. Hence wrapping it in python. – Mike Addison Jul 20 '16 at 16:20

1 Answers1

0

Never mind, I figured it out. Too much coffee and not enough sleep. The answer is to use: goldkeyID=open('goldenClusterID', 'r').read() os.system("aws emr describe-cluster --cluster-id %s" % goldkeyID)

Mike Addison
  • 23
  • 6