Stack Overflow
Stack Overflow

Questions tagged [external-tables]

To be used for External tables. These tables don't necessarily have data in the data warehouse(or database directory).

Hive

An EXTERNAL table points to any HDFS location for its storage, rather than being stored in a folder specified by the configuration property hive.metastore.warehouse.dir(Location of default database for the warehouse). See also official Hive docs on Managed vs External Tables

Oracle

External tables allow Oracle to query data that is stored outside the database in flat files. The ORACLE_LOADER driver can be used to access any data stored in any format that can be loaded by SQL*Loader. No DML can be performed on external tables but they can be used for query, join and sort operations.

381 questions
4
votes
1 answer

what is RELY constraint in hive external tables?

managed table vs external table in above link they mentioned "Only the RELY constraint is allowed on external tables." But no info that what is RELY constraint. Can somebody help me to understand what is Only the "RELY constraint". thanks in…
Pala
  • 2,011
  • 3
  • 15
  • 17
4
votes
2 answers

Oracle 11g External Tables size limit

Is there a limit for the files that are defined as external tables in Oracle 11g? As per http://download.oracle.com/docs/cd/B19306_01/server.102/b14237/limits002.htm, the last parameter External Tables file - Maximum size, it is Dependent on the…
Shyam Kumar Sundarakumar
  • 5,649
  • 13
  • 42
  • 69
4
votes
1 answer

Hive Insert overwrite into Dynamic partition external table from a raw external table failed with null pointer exception.,

I have a raw external table with four columns- Table 1 : create external table external_partitioned_rawtable (age_bucket String,country_destination String,gender string,population_in_thousandsyear int) row format delimited fields terminated…
Barath
  • 107
  • 2
  • 14
4
votes
2 answers

Getting an external table's location and file name?

I'm processing multiple files as part of an external table. Is there any way I can get the file name being processed in external tables and place it in the table? At the moment the only solution I can find is appending the file name to every record…
Matt Damon
  • 323
  • 1
  • 6
  • 18
4
votes
2 answers

External Tables (HIVE) Choose only a few columns from a file

How can I create a external table setting only a few columns from a file? Ex: In archive I have six columns, A,B,C,D,E,F. But in my table i want only A, C, F. Is It possible?
Elder Chaves
  • 41
  • 1
  • 2
4
votes
2 answers

How to skip file headers in impala external table?

I have file on HDFS with 78 GB size I need to create an Impala External table over it to perform some grouping and aggregation on data available Problem The file contain headers. Question Is there any way to skip headers from file while reading the…
Vikas Hardia
  • 2,635
  • 5
  • 34
  • 53
3
votes
1 answer

Bigquery dbt_external_tables External Data Configuration

I need some help when using the dbt_external_tables package. I realized that in the csv I have in GCS some lines appear to have line breaks and this is causing some issues when trying to query the table created by the macro. Sometimes when doing…
Tomas Conticello
  • 53
  • 2
  • 10
3
votes
1 answer

Bigquery export splits into multiple files with some empty files

I am trying to use the bigquery export functionality to push data out to GCS in json format. At the end of the process inorder to validate the count of exported records in the GCS file, I am creating an external table with auto schema detection…
jay12345
  • 31
  • 2
3
votes
1 answer

Reading the same Oracle external table multiple times with the different underlying file content raises the error KUP-05011: Size of file has changed

I try to read the same Oracle external table multiple times with the different underlying file content. I use the java app with the code fragment like: int sptepn = 1; for (File file: files) { log_step(stepn++); …
igortche
  • 115
  • 1
  • 8
3
votes
2 answers

Azure Synapse - String Delimiter

I have a text file with the following format. "01|""sample""|""Test"|""testing""|""01"|"""". I have created an external table in Azure Synapse by setting the format option STRING_DELIMITER to '"'. But while processing the file through an sp, i am…
Sandeep T
  • 421
  • 8
  • 22
3
votes
1 answer

Understanding Azure SQL Server External Tables

We are trying to create a cross-database query using Azure's preview Elastic Query. So we will be creating an External Table to make these queries happen. Unfortunately, I have some apprehension about how the queries will be executed. I don't want a…
CompyKelly
  • 51
  • 1
  • 3
3
votes
1 answer

Confusion with the external tables in hive

I have created the hive external table using below command: use hive2; create external table depTable (depId int comment 'This is the unique id for each dep', depName string,location string) comment 'department table' row format delimited fields…
RushHour
  • 494
  • 6
  • 25
3
votes
1 answer

How to create n number of external tables with a single hdfs path using Hive

Is it possible to create n number of external tables are pointing to a single hdfs path using Hive. If yes what are the advantages and its limitations.
rajendra moka
  • 31
  • 1
  • 2
3
votes
1 answer

Changing column datatype in parquet file

I have an external table pointing to an s3 location (parquet file) which has all the datatypes as string. I want to correct the datatypes of all the columns instead of just reading everything as a string. when i drop the external table and recreate…
querymind
  • 31
  • 1
  • 2
3
votes
1 answer

Create External Table in Azure SQL Data warehouse to a wild card based file or folder path

I know we can create an External table in Azure SQL Data warehouse pointing to a LOCATION that is either a file path or a folder path. Can this file or folder path be based on a wild card pattern instead of an explicit path. Here my file path is a…
Shahzad Badar
  • 83
  • 1
  • 5
1
2
3
…
25 26