3

I'm trying to test restoring Neptune instances from a snapshot using python (boto3). Long story short, we want to spin up and delete the Dev instance daily using automation.

When restoring, my restore seems to only create the cluster without creating the attached instance. I have also tried creating an instance once the cluster is up and add to the cluster, but that doesn't work either. (ref: client.create_db_instance)

My code does as follows, get the most current snapshot. Use that variable to create the cluster so the most recent data is there.

import boto3

client = boto3.client('neptune')

response = client.describe_db_cluster_snapshots(
    DBClusterIdentifier='neptune',
    MaxRecords=100,
    IncludeShared=False,
    IncludePublic=False
)

snaps = response['DBClusterSnapshots']
snaps.sort(key=lambda c: c['SnapshotCreateTime'], reverse=True)

latest_snapshot = snaps[0]
snapshot_ID = latest_snapshot['DBClusterSnapshotIdentifier']

print("Latest snapshot: " + snapshot_ID)

db_response = client.restore_db_cluster_from_snapshot(
    AvailabilityZones=['us-east-1c'],
    DBClusterIdentifier='neptune-test',
    SnapshotIdentifier=snapshot_ID,
    Engine='neptune',
    Port=8182,
    VpcSecurityGroupIds=['sg-randomString'],
    DBSubnetGroupName='default-vpc-groupID'
)

time.sleep(60)

db_instance_response = client.create_db_instance(
    DBName='neptune',
    DBInstanceIdentifier='brillium-neptune',
    DBInstanceClass='db.r4.large',
    Engine='neptune',
    DBSecurityGroups=[
        'sg-string',
    ],
    AvailabilityZone='us-east-1c',
    DBSubnetGroupName='default-vpc-string',
    BackupRetentionPeriod=7,
    Port=8182,
    MultiAZ=False,
    AutoMinorVersionUpgrade=True,
    PubliclyAccessible=False,
    DBClusterIdentifier='neptune-test',
    StorageEncrypted=True
)

The documentation doesn't help much at all. It's very good at providing the variables needed for basic creation, but not the actual instance. If I attempt to create an instance using the same Cluster Name, it either errors out or creates a new cluster with the same name appended with '-1'.

LiquidCourage11
  • 43
  • 3

1 Answers1

0

If you want to programmatically do a restore from snapshot, then you need to:

  1. Create the cluster snapshot using create-db-cluster-snapshot
  2. Restore cluster from snapshot using restore-db-cluster-from-snapshot
  3. Create an instance in the new cluster using create-db-instance

You mentioned that you did do a create-db-instance call in the end, but your example snippet does not have it. If that call did succeed, then you should see an instance provisioned inside that cluster.

When you do a restore from Snapshot using the Neptune Console, it does steps #2 and #3 for you.

It seems like you did the following:

  1. Create the snapshot via CLI
  2. Create the cluster via CLI
  3. Create an instance in the cluster, via Console

Today, we recommend restoring the snapshot entirely via the Console or entirely using the CLI.

The-Big-K
  • 2,672
  • 16
  • 35
  • Thanks. Working on the snapshot code now. It was going to be the second phase for the 'shutdown' portion of the Dev environment. I'll update later this afternoon. That said, my process was to just create the snapshot manually via console just for testing to validate the name, etc. so I could validate some things in my code. Again, thanks! I'll update shortly on what happens and where I land on this. I've edited my original comment to show the additional code creating the instance. – LiquidCourage11 Oct 19 '18 at 13:00
  • So, there are some issues. I'm having issues getting the instance added to the cluster upon creation. I'm getting the following error based on my code in the original post: `botocore.exceptions.ClientError: An error occurred (InvalidParameterCombination) when calling the CreateDBInstance operation: The requested DB Instance will be a member of a DB Cluster. Set database name for the DB Cluster.` . However, I used the 'DBClusterIdentifier' with the exact cluster name. In theory, I should be able to add an instance or additional instances using this, correct? – LiquidCourage11 Oct 19 '18 at 16:41
  • Yes, you should be able to add instances like this. Can you try removing "DBName" from your arguments to CreateDbInstance? – The-Big-K Oct 19 '18 at 16:55