Questions tagged [snowflake-schema]

**Snowflake schema** is special case of the database Star Schema, where one or many dimension tables are normalized. This is often done for improving the performance in some cases of the Star Schema. When the dimension tables are completely normalized, the resulting structure resembles a snowflake with the fact tables in the middle.

Snowflake schema is a logical arrangement of tables in a multidimensional database in which the dimensions are expressed by more than one table. The dimension tables that relate to the facts table represent the smallest granularity of data-analysis. The other tables in the same dimension represent aggregators of dimension's individual elements. When a dimension is aggregated by several independent criteria, the diagram resembles the shape of a snowflake. Used in data warehouses.

Methods for creating Snowflake Schema out of Star Schema:

  • Partially or fully normalize one or many dimension tables, leaving other dimensions unchanged.
  • Partially or fully normalize all dimension tables.

When to use

Query performance takes the highest significance in a data warehouse and snowflaking can hamper the performance. It very useful for cases, when some attributes like demographic attributes are browsed more then the whole customer table itself. Another case is for precalculated summaries derived from a fact table. This helps not aggregating over and over again since the operation is expensive.

Advantages

Savings in the storage space. Normalized structures are having no insertion, deletion or update anomalies. Easier to update and maintain.

Disadvantages

Schemas are less intuitive, with higher complexity and abstraction. Difficult browsing through the contents for users. Additional joins - worse query performance in some cases.

Example

Having a fact table called "Sales" and dimension table called "Product". Assume there are 500000 product dimension rows. These products can fall under 10 brands. For running query for products, not indexed on brands, the query will have to search through 500000 rows to find all brands.

On the other hand, if the "Product" table partially normalized, separated on another table "Brand" the initial search query would need to go through just 10 rows on.

External links

893 questions
0
votes
1 answer

Snowflake: Fail COPY INTO incase of error limit while loading?

Is it possible to set the error limit while loading the data into a snowflake table. I am using COPY INTO option. I know there are options like RETURN_FAILED_ONLY and VALIDATION_MODE, but this does not support if the error limit is reached then fail…
Vishrant
  • 15,456
  • 11
  • 71
  • 120
0
votes
1 answer

How to use regex while selecting rows from snowflake external stage?

I have created following things: External Stage (pointing to folder in Data Lake Gen2) File Format (Parquet) And I want to do a select from a stage but I want to read only those files that have .part. in there filename. Using copy into command I…
0
votes
2 answers

COPY INTO: is there a way to show number of records skipped during loading data into Snowflake?

I am using copy-into-table from an external location and there is an option to continue loading the data in case the row have corrupted data. Is there an option to show how many rows were skipped while loading, like there is an option in Teradata…
Vishrant
  • 15,456
  • 11
  • 71
  • 120
0
votes
1 answer

Load data into snowflake table which has more than 1 column from json file

I got this link from snowflake site "https://youtu.be/H0sbMDqdYQ8" ,were they are trying load from json file with copy command ,where the table has 4 column of the 2 are variant.i am trying the same ,but when try load the json file with copy command…
0
votes
2 answers

How to unload data from Snowflake in parquet format with lowercase column name?

I have to unload the Snowflake data into external s3 location in the parquet format in the lowercase column name. By default it's coming as uppercase, is there a way of doing it? Update: Following is the command to create view: create view test_view…
Vishrant
  • 15,456
  • 11
  • 71
  • 120
0
votes
1 answer

Snowflake: Failed with exception java.io.IOException:org.apache.parquet.io.ParquetDecodingException: Can not read value at 0 in block -1

After unloading the data in parquet format to s3 from Snowflake, then I am creating a Hive table with the equivalent snowflake schema, with s3 as the external location, but I am getting the following error: Failed with exception…
Vishrant
  • 15,456
  • 11
  • 71
  • 120
0
votes
1 answer

How to write MS unit test for Snowflake using C#

I am using Snowflake Connector to communicate my .NET Core application with Snowflake. My requirement is to write MS unit tests for the repository layer My repository class looks like: using (IDbConnection conn = new SnowflakeDbConnection()) …
0
votes
1 answer

snowflake special character parsing issue searching for solution

I have a field name called question and value in that field is How do you think of ${question_1}? I want to write a query to select this value and I think it's going to use regular expression to do it. Below is the query but it didn't get the…
xmz
  • 151
  • 4
  • 14
0
votes
3 answers

Couldn't load data into snowflake

I am trying to load csv data file into snowflake. But it shows this error "Numeric value '0xA1T0xA920xB4' is not recognized File '@TRY/ui1592405587937/New Microsoft Excel Worksheet.csv', line 2, character 1 Row 1, column "TRY"["DEC":1] " My csv…
0
votes
2 answers

How to Update Table in Snowflake using Azure Data Factory

I have two tables in snowflake named table1 and table2. Table1 is the source table which contains incremental data and table2 is the target table. So my usecase is I have to take data from table1 and update the data into table2 but this process has…
0
votes
1 answer

Snowflake unable to load data with double quote in Data as well as for test Values

Below is Data "1"|"2"|"XCEFFDDD - "CV" test"|"3" I can't change the input data, with the above data when ingesting to snowflake using the FileFormat with options FIELD_DELIMITER = '|', ESCAPE_UNENCLOSED_FIELD = 'NONE', TRIM_SPACE = TRUE, NULL_IF =…
0
votes
2 answers

Load data from multiple filetypes into same snowflake table?

Can I use multiple file-formats with different filetype to load data into same snowflake table, I read this documentation, but it does not specify explicitly. There are some files that I load from the external stage (s3) to snowflake that are in…
Vishrant
  • 15,456
  • 11
  • 71
  • 120
0
votes
1 answer

Adding current date in create script for Snowflake

I have a requirement where I have to create tables with the date/datetime in the table name when they were created dynamically.Wondering if this option is possible in Snowflake? Eg: I would need somethinglike this. CREATE TABLE…
0
votes
1 answer

Need help parsing this in snowflake

[ { "legs": [ { "points": [ { "latitude": 29.76524, "longitude": -95.35406 }, { "latitude": 30.74438, "longitude": -101.71203 }, { "latitude": 30.74421, "longitude": -101.71247 }, { "latitude": 30.74404, "longitude": -101.71289 }, { "latitude":…
0
votes
1 answer

SnowSql no response

I am using SnowSql 1.2.5 and I receive no response, no error, absolutely nothing from it. It does not matter which command I type, even if type just "SnowSql.exe" and hit confirm, I get no response. This is happening in an specific machine, in…