I using the https://docs.aws.amazon.com/sdkfornet/v3/apidocs/items/Athena/NAthena.html sdk nuget https://github.com/aws/aws-sdk-net/ in order to query S3 bucket files, the thing is I need to create dataCatalog/databases and tables per demand (inside my program), I found a method to create dataCatalogs, but there is no such thing for databases or tables, only query methods. I'm confusing and want to know how can I do such thing.
Asked
Active
Viewed 462 times
1 Answers
0
Indeed there is no specific method for creating databases and tables, for that you will need to use QueryExecution requests
For databases:
var query = $"CREATE DATABASE IF NOT EXISTS DatabaseName " +
"COMMENT 'some comment' " +
"WITH DBPROPERTIES ('creator'='Me', 'Extra.'='some extra info');";
var request = new StartQueryExecutionRequest
{
QueryString = query,
ResultConfiguration = new ResultConfiguration
{
OutputLocation = _settings.DDLOutputLocation
}
};
For tables:
var query = "CREATE EXTERNAL TABLE IF NOT EXISTS " +
"TableName ( " +
"column1 int, " +
"column2 string, " +
"column3 int, " +
"column4 binary) " +
"ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' " +
$"LOCATION '{_settings.TableDataSourceLocation}'";
var request = new StartQueryExecutionRequest
{
QueryString = query,
QueryExecutionContext = new QueryExecutionContext
{
Catalog = _settings.CatalogName,
Database = _settings.DatabaseName
},
ResultConfiguration = new ResultConfiguration
{
OutputLocation = _settings.DDLOutputLocation
}
};

Andre
- 652
- 2
- 7
- 23