Questions tagged [database-metadata]

Metadata is "data about data". In database terminology, this set of metadata is referred to as the catalog. The SQL standard specifies a uniform means to access the catalog, called the information schema, but not all databases implement it, even if they implement other aspects of the SQL standard. For an example of database-specific metadata access methods, see Oracle metadata.

Meta Data:

Metadata is “data about data”. An item of metadata describes the specific characteristics about an individual data item. In databases, metadata describes the structural components of tables and their elements. For example, metadata about an element could include data types, name of data, size and many more characteristics about that element. It would also give information about the tables the database is storing, information, such as length of fields, number of columns, where the tables are located and other pertinent information. One of the main uses for met data is to provide a link between the information creator and the information users. Metadata allows the users to speed up the search for individual data. This is done by being able to set parameter for searches, allowing the filtering of unwanted information. Metadata can be stored either internally, in the same file as the data or externally, in a separate area. Both have advantages and disadvantages. If the data is stored internally, the metadata is together with the data making more easily accessible to view or change. However, this method creates high redundancy. If metadata is stored externally, the searches can become more efficient. There is no redundancy but getting to this metadata may be a little more technical. There are certain formats that must be used, such as Uniform Resource Identifier(URI) to get to the metadata if this format is not used the metadata becomes inaccessible. All the metadata is stored in a data dictionary or a system catalog. All programs that access data in the database work through a DBMS. The DBMS uses the data dictionary to look up the required components and relationships. Any changes made to the database structure are automatically recorded in the data dictionary. This makes the data dictionary manager’s job a lot easier because any modification of programs that are affected by the changed structure is not necessary.

Metadata at the most basic level is simply defined as “data about data”. An item of metadata describes the specific characteristics about an individual data item. In the database realm, metadata is defined as, “data about data, through which the end-user data are integrated and managed.” (Rob & Coronel, 2009) Metadata in a database typically store the relationships that link up numerous pieces of data. “Metadata names these fields describes the size of the fields, and may put restrictions on what can go in the field (for example, numbers only).” (Sheldon, 2001).

“Therefore, metadata is information about how data is extracted, and how it may be transformed. It is also about indexing and creating pointers into data. Database design is all about defining metadata schemas.” (Sheldon, 2001) Metadata can be stored either internally, in the same file as the data, or externally, in a separate area. If the data is stored internally, the metadata is together with the data, making it more easily accessible to view or change. However, this method creates high redundancy. If metadata is stored externally, the searches can become more efficient. There is no redundancy but getting to this metadata may be a little more technical.

All the metadata is stored in a data dictionary or a system catalog. The data dictionary is most typically an external document that is created in a spreadsheet type of document that stores the conceptual design ideas for the database schema. The data dictionary also contains the general format that the data, and in effect the metadata, should be. Metadata is an essential aspect of database design, it allows for increased processing power, due to the fact that it can help create pointers and indexes.

Useful Resources:

217 questions
0
votes
1 answer

How emulate PDO::getColumnData() using Oracle?

I have a vanilla PHP 7 app that (aims to) suports MySQL, Postgres, SQL Server and Oracle. I've faced a problem now because all but Oracle (pdo_oci), supports getColumnData() $statement = $pdo->query('SELECT titulo, preco FROM livro'); $metadados …
celsowm
  • 846
  • 9
  • 34
  • 59
0
votes
1 answer

Why Spring Batch reads metadata tables from MASTER and not from User defined Schema?

I have a spring boot batch application. In application.properties, I specify my data source details as…
User_1940878
  • 321
  • 1
  • 6
  • 25
0
votes
1 answer

PSQLException: ERROR: column "attidentity" does not exist

I'm using PostgresSQL 9.6 and have a process that is introspecting the database. I have the following error: org.postgresql.util.PSQLException: ERROR: column "attidentity" does not exist Position: 169 at…
Christian68
  • 845
  • 4
  • 13
  • 24
0
votes
1 answer

Do system table modifications pass through replication?

I am currently patching a Postgres 9.6 database. As part of the patch I'm required to make changes to the system tables, first changing allow_system_table_mods in the config file, then executing the recommended SQL. If I make this change on a…
VynlJunkie
  • 1,953
  • 22
  • 26
0
votes
1 answer

SQLAlchemy: how to create dynamic columns multiple times

I succeeded to create a sql table with columns defined dynamically, thanks to python class reflexion. But I cannot run the code more than one time. For instance, the following import_file , should create a static table and a dynamic table with…
DrIDK
  • 7,642
  • 2
  • 14
  • 14
0
votes
0 answers

Hive metadata view row count

How to get view count of rows from metadata. Joining TABLE_PARAMS on TBLS: SELECT * FROM hive.TABLE_PARAMS tb JOIN hive.TBLS t ON t.TBL_ID = tb.TBL_ID WHERE tb.PARAM_KEY = 'numRows' AND t.TBL_NAME like 'view_%' doesn't work as view is not a…
0
votes
1 answer

SQL Server MetaData NVarChar size

One of our SQL Server column was recently modified from varchar to nvarchar, as a result the max_length value in sys.tables doubled. In the vb counterpart, our SQLMetadata objects were instantiated using the character length of the column, the last…
Orion
  • 39
  • 3
0
votes
1 answer

DB2 table's UNIQUE INDEX not showing in TABCONST and CONSTDEP tables

I am trying to query the unique indices from SYSCAT tables for some user tables using the query below which uses 4 catalog tables (INDEXES, INDEXCOLUSE, TABCONST, CONSTDEP). I realized all indices are there in INDEXES and INDEXCOLUSE tables but many…
CamelCamelius
  • 333
  • 2
  • 15
0
votes
1 answer

How to return a scrollable result set from getMetaData.getColumns()?

I need to return a scrollable result set from the code below: ResultSet columnMetaData = connection.getMetaData().getColumns(null, null, "my_table", "%"); However, the getColumns() API doesn't appear to provide a way for me to say what kind of…
jones-chris
  • 629
  • 2
  • 13
  • 29
0
votes
1 answer

DBeaver: column description missing for DB2 Host

We are using DBeaver with a DB2 host database. Table descriptions get displayed just fine. But for table columns there is no description under table properties→columns. SELECT name,tbname, LABEL FROM SYSIBM.syscolumns yields the correct/desired…
marcus
  • 1
  • 4
0
votes
1 answer

MongoDB tags with key value pair search

I have a mongo collection 'CARDS' having a list of tag objects. (kind of metadata) "tags" : [ { "tagValue" : "retail", "tagName" : "CARD_TYPE" }, { "tagValue" : "fast food ", …
Jayanta Baishya
  • 97
  • 1
  • 12
0
votes
1 answer

Extract metadata from database using python?

I have looked for an answer to this question, but very was able to find very little. I want to extract the names of the tables, references between them, column names so I can graphically visualize that information. I need to this in a Django…
Neven Jovic
  • 177
  • 1
  • 2
  • 8
0
votes
0 answers

DatabaseMetadata.getImportedKeys() throws Column 'REFERENCED_TABLE_NAME' in where clause is ambiguous when useInformationSchema is true

I am trying retrieve keys for a table on a MySQL database using DatabaseMetaData and trying to use InformationSchema to do it. MySQL Configuration Properties import java.sql.* class DataLoader { static final String JDBC_DRIVER =…
MajorXbox
  • 503
  • 1
  • 4
  • 13
0
votes
1 answer

Talend Studio - Search inside Metadata to get all the queries created

I'm using Talend Studio, I've a lot of jobs created each one with multiple queries. I want to to search for all the jobs that have the SQL Statement "Insert Into". There exists any place (like a database) where Talend save the commands and we can…
SaCvP
  • 393
  • 2
  • 4
  • 16
0
votes
0 answers

Business Intelligence : Data warehouse creation (ORACLE)

I am working on a business intelligence module. Since I don't have metadata to create my data warehouse , Is it possible to find a script or any other solution to create that metadata from a simple 2 dimensions database ? PS: I'am using ORACLE…