Stack Overflow
Stack Overflow

Questions tagged [partitioning]

Partitioning is a performance strategy whereby you divide possibly very large groups of data into some number of smaller groups of data.

Partitioning is a performance strategy whereby you divide possibly very large groups of data into some number of smaller groups of data.

The expectation is that with algorithms of order exponentially greater than N the total time it takes to process the smaller groups and combine the results is still less than the time it would take to process the one larger set of data.

Partitioning is similar to range partitioning in many ways. As in partitioning by RANGE, each partition must be explicitly defined.

3138 questions
1
vote
1 answer

BEFORE Trigger + RETURNING returns NULL

This is one of the BEFORE triggers that inserts into the right table partition: CREATE OR REPLACE FUNCTION public.insert_install_session() RETURNS trigger LANGUAGE plpgsql AS $body$ BEGIN IF (NEW.created >= '2015-10-01 00:00:00' AND…
kev
  • 8,928
  • 14
  • 61
  • 103
1
vote
1 answer

How to change RANGE value in PARTITIONS BY Range Oracle

I have an existing table within the next script: create table sales6 ( sales_id number, sales_dt date ) partition by range (sales_dt) ( partition p0701 values less than…
ZetaPR
  • 964
  • 7
  • 32
1
vote
2 answers

How Oracle execute a select on a partitioned table with a date range criteria

currently we have a working query that selects from a large partitioned table for processing. The table is partitioned by daily range, with 1 day of data in 1 partition. The query is: SELECT /*+parallel(auto)*/ a.* FROM TBL_EXCLUDED a …
ipohfly
  • 1,959
  • 6
  • 30
  • 57
1
vote
0 answers

Partitioning a group of people into duets and/or singles, how do I find all possible configurations of partitions?

So I have a group of people, typically between 15 and 30 people. I have a class Person to represent every person. I also have a class Match with represents a duo with its properties $person1 and $person2, which incidentally can also both point to…
Evert
  • 2,022
  • 1
  • 20
  • 29
1
vote
1 answer

Partitioning by specific rows

What we have is this, a table sorted by date and time: date | time | amount | type -----------+-------------+--------+------- 11/09/2014 | 11:13:03 AM | 1 | USE 11/09/2014 | 11:14:03 AM | 2 | USE 11/09/2014 | 12:13:03 AM | 10 …
Felix Wolff
  • 11
  • 1
1
vote
1 answer

Can the partition number of a Spark RDD be manually changed without repartitioning

In Spark I have two PairRDDs (let us call them A and B) consisting of n partitions each. I want to join those RDDs based upon their keys. Both RDDs are consistently partitioned, i.e., if keys x and y are in the same partition in RDD A, they are also…
Philosophus42
  • 472
  • 3
  • 11
1
vote
1 answer

Joining unequal sized data sets in spark

I have the following data sets: Dataset 1: Dataset 2: Dataset 3: id field1 l_id r_id id field2 Here are their sizes: Dataset1: 20G Dataset2: 5T Dataset3: 20G Goal: I would like…
soontobeared
  • 441
  • 4
  • 9
  • 30
1
vote
4 answers

Faster way to load huge data warehouse table

I have a table in oracle 11g with size 62GB and 1.2 billion records. Table has 4 columns, range partitioned on month and hash sub partition on transaction number. Around 80 million records are delated and re-inserted into this table once in every…
Vivek
  • 4,452
  • 10
  • 27
  • 45
1
vote
3 answers

How to change the number of partitions of an RDD with a large local file (non-HDFS file)?

I have a 8.9GB text file and I create an RDD out of it and imported it in Spark. textfile = sc.textFile("input.txt") The number of partitions that Spark creates is 279, which is obtained by dividing the size of the input file by 32MB default HDFS…
MPAK
  • 39
  • 2
  • 5
1
vote
1 answer

ORACLE - Partitioning with changing values

Assuming following table: create table INVOICE( INVOICE_ID NUMBER ,INVOICE_SK NUMBER ,INVOICE_AMOUNT NUMBER ,INVOICE_TEXT VARCHAR2(4000 Char) ,B2B_FLAG NUMBER -- 0 or 1 …
royskatt
  • 1,190
  • 2
  • 15
  • 35
1
vote
1 answer

ASP.net Partition SQL

I am developing an ASP.Net application using SQL Server. Looking up on forums I found that the best practice to deal with big databases is to use indexes and partitions. But I didn't find anywhere how to do this programmatically in application. I…
porandddr
  • 129
  • 1
  • 1
  • 6
1
vote
1 answer

MySQL: Indexing Table With 10+ Million Rows

I have a table in my database with 10+M rows. Actually I never worked with so many records and it cames across me that I need a little help on indexing / partitioning the table. The table looks like this: CREATE TABLE `stock` ( `ID` bigint(20) NOT…
Alberto Ar3s
  • 311
  • 1
  • 3
  • 14
1
vote
2 answers

Calculate % using OVER() for each row in SQL Server 2012

I needed guidance with a question on how to calculate percentage between two cells on each row. I am on SQL Server 2012. Below is the structure of my current dataset. Customer_Number Price_Last_Year …
user3197575
  • 269
  • 6
  • 13
1
vote
1 answer

why is it very slow to do query on master table?

I have a very large table. So I partitioned this table. And copied all records from master table to child tables. Then I deleted all records from master table. Now my master table is empty. It is very fast to do query on child table. But it is still…
old bird
  • 67
  • 5
1
vote
1 answer

VB.NET - Combine TakeWhile and SkipWhile to get both partitions

Is there a way to combine a TakeWhile and SkipWhile on the same list using the same predicate to get both partitions, or would that be premature optimization? Would this be done automatically behind the scenes? Note that the list is sorted by…
mbomb007
  • 3,788
  • 3
  • 39
  • 68
1 2 3
…
99
100