Stack Overflow
Stack Overflow

Questions tagged [partitioning]

Partitioning is a performance strategy whereby you divide possibly very large groups of data into some number of smaller groups of data.

Partitioning is a performance strategy whereby you divide possibly very large groups of data into some number of smaller groups of data.

The expectation is that with algorithms of order exponentially greater than N the total time it takes to process the smaller groups and combine the results is still less than the time it would take to process the one larger set of data.

Partitioning is similar to range partitioning in many ways. As in partitioning by RANGE, each partition must be explicitly defined.

3138 questions
1
vote
1 answer

Create partition in an indexed table

I have a table which holds data for 12 hours. Every 5 minutes, it keeps deleting data which is more than 12 hours old and adds new data. It has almost 15-20 million rows. I want to create partition by hour and also index the table on…
Jaydeep
  • 149
  • 2
  • 5
  • 19
1
vote
1 answer

Spring Batch JDBCPagingItemReader not partitioning equally for each thread

this is my first question here. I am working on a spring batch and I am using step partitioning for processing 70K records. For testing I am using 1021 records and found that the partitioning not happening equally for each thread. I am using…
Anup Ghosh
  • 332
  • 2
  • 6
  • 23
1
vote
2 answers

How to scale out by evolving from database partitions to sharding?

Say I have a MySQL table: CREATE TABLE tweets ( tweet_id INT NOT NULL AUTO_INCREMENT, author_id INT NOT NULL, text CHAR(140) NOT NULL, PRIMARY KEY (tweet_id) ) PARTITION BY HASH(tweet_id) PARTITIONS 12; All is good. The table lives on a single…
Continuation
  • 12,722
  • 20
  • 82
  • 106
1
vote
2 answers

Error "A PRIMARY KEY must include all columns in the table's partitioning function"

When adding partitioning index on the column country_id to the following table, i get the error: A PRIMARY KEY must include all columns in the table's partitioning function I tried to add the column country_id to the PK containing then id and…
delete
  • 18,144
  • 15
  • 48
  • 79
1
vote
1 answer

MySQL - rebuild partition vs optimize partition

I've partitioned tables in my MySQL 5.1.41 which hold very huge amount of data. Recently, I've deleted a lot of data which caused fragmentation of around 500 GB yet there is a lot of data in the partitions. To reclaim that space to the OS, I had to…
Yashwanth Aluru
  • 1,125
  • 6
  • 21
  • 28
1
vote
1 answer

Create partition if not exist on MySQL

I have the following table id | p_key | col3 | col4 ------------------------ 1 | pr1 | c1 | co1 2 | pr2 | c2 | co2 3 | pr3 | c3 | co3 Now I have to create partition on project key, something like If partition partpr1 not…
Md. Parvez Alam
  • 4,326
  • 5
  • 48
  • 108
1
vote
1 answer

Delete large amount of data using partitioning in mysql

I have a table that is constantly growing. I want to delete rows that are older than 1 year (periodically - each 12 hours) At first I thought using the ordinary delete statement, but it's not good as there are many entries and the database will get…
user1386966
  • 3,302
  • 13
  • 43
  • 72
1
vote
1 answer

MySQL Partitioning user generated rows by user

I have two tables: userMessages and userStatistics I realized that I need to set up a partitioning in order to ensure efficiency. With all the information I could gather, I am suppose to use HASH partitioning. PARTITION BY HASH(user_id) PARTITIONS…
Maciek Semik
  • 1,872
  • 23
  • 43
1
vote
1 answer

How to query against several tables from a partition in PostgreSQL

I have a big table partition in PostgreSQL by disjoint conditions. Something like: CREATE TABLE child_table_1(check(my_condition = '01')) INHERITS (parent_table); // ... CREATE TABLE child_table_20(check(my_condition = '20')) INHERITS…
Gabriel Furstenheim
  • 2,969
  • 30
  • 27
1
vote
2 answers

SQL interpolate missing dates

Using SQL Server 2012 I have a table called Allbucket CustodianAccountNum symbol EndDate ManagerName MarketValue NetReturn A9G040819 wabix 12/31/2013 GMO Benchmark 34751.10987 0.004072 A9G040819 wabix 1/31/2014 GMO Benchmark …
Jay C
  • 842
  • 6
  • 17
  • 37
1
vote
0 answers

SQL Server 2014 Partitioning

I need some help figuring out how to best partition a large transaction table in SQL Server 2014. The transaction table has a CLUSTERED INDEX on the date, a primary key to identify each record, and an institution ID to identify which institution…
Keith Harris
  • 1,118
  • 3
  • 13
  • 25
1
vote
1 answer

Does using Hive columns/partitions that are subsets of each other improve query performance?

I'm working with a Hive table that is partitioned by year, month, and day. e.g. year=2015 AND month=201512 AND day = 20151231. From my limited knowledge of the way Hive works, these are probably set up in a folder structure where the '2015' folder…
bsg
  • 825
  • 2
  • 14
  • 34
1
vote
1 answer

Data Partition in R

I am looking for a robust way to partition a dataset without using the sample() function, and hope to get some feedback. As a matter of fact, I'd ideally like to get rid of the of random property inherent to the usage of…
owner
  • 723
  • 3
  • 9
  • 25
1
vote
0 answers

Tablespaces with interval partitioning

Is there any way to specify different tablespaces for each partition if I use interval partitioning? My table is (it is just example, I have more columns in the table): create table MY_TABLE ( id NUMBER(20) not null, type …
Tatiana
  • 1,489
  • 10
  • 19
1
vote
1 answer

Table partition

I need to prepare the script to increase the partition range if the partition is going to get finished in next 2-3 months. How to find the existing table partition and we can edit to existing table or we need to create a new script. Appreciate…
junaid
  • 11
  • 2
1 2 3
…
99 100