Bulk update a database
Methods to bulk update a database with single or multiple columns of data. This could be either unique data for each row or same data for a column.
Questions tagged [bulkupdate]
299 questions
0
votes
1 answer
bulk update not working for filefield upload django
I am trying to update multiple files in fileField in django. I am fetching the objects from the database and i am assigning the new file by iteration over them and then, saving the object in the list. If i use bulk_update it updates all the fields…
Arsh Doda
- 294
- 4
- 14
0
votes
1 answer
How to implement bulk updateOne with mongodb in java
I'm trying to implement bulk update for a list of objects. But I'm getting the following error
java.lang.IllegalArgumentException: Invalid BSON field name bookingId
at org.bson.AbstractBsonWriter.writeName(AbstractBsonWriter.java:532)
at…
Nischit Pradhan
- 440
- 6
- 18
0
votes
1 answer
Post 800 json files to Shopify API (limitation is 40/second)
We've about 800 json files to be posted to a Shopify API. It can't be merged into a single API call (there's no BULK post API). Shopify limits 40 requests per second. This is the API we're posting data :
POST…
Hemant Tank
- 1,724
- 4
- 28
- 56
0
votes
1 answer
Is it possible to bulk_update a many-to-many relationship in django 2.2?
I have these two simple models:
class school(models.Model):
name = models.CharField(max_length=128)
class teacher(models.Model):
schools = models.ManyToManyField(School)
user = models.OneToOneField(...)
In a view, let's say I want to…
Flexonze
- 27
- 1
- 6
0
votes
1 answer
Bulk Whois lookup Of 20,000 domains - getting timeouts
Am trying to BULK extract WHOIS information for 20,000 domain names, the python code works with 2 items in my csv file but brings error with the whole dataset of 20000 domain names
tried with 2 domain names, OK. using a full list of 20k domain names…
Sam
- 5
- 3
0
votes
0 answers
Update 62Millions Records in Oracle
I have to update 62 millions of records in production database. Its a simple update statement.
Its a pretty big table.
This is the total number of records count in that table = 1251797271.
Can I approach bulk collect method for the updating the…
Shivaram
- 11
- 3
0
votes
0 answers
How to fetch data after INSERT and UPDATE queries in one TRANSACTION
What I have is an array of objects and I need to update the DB layer based on that array.
If the object contains id property - Update query, if it doesn't - Insert query.
What I end up with is
BEGIN TRY
BEGIN TRANSACTION
UPDATE {...}
INSERT INTO…
Pavlin Petkov
- 1,022
- 3
- 17
- 38
0
votes
1 answer
How to eliminate ambiguous column reference error in bulk upsert for postgresql?
For the following code I am trying to do a bulk upsert into a table. The table is called jobstep_to_step_relationships. If the primary key column "job_base_step_id" already exists in the table I want the algorithm to do an update instead.
create…
Brian Yeh
- 3,119
- 3
- 26
- 40
0
votes
0 answers
simple update (SQL) vs bulk update (PL/SQL)
Recently I came across an interview question about :
which is a better and faster was of updating a table in oracle DB for more than 1 Million records and why ?
simple update :
update employee
set salary = salary + (salary*5%)
where dept_id =…
Biswajit Buragohain
- 47
- 7
0
votes
1 answer
Does mongodb lock between bulk groups?
Given a collection containing documents of the following format:
{
name:String,
members: [ { name:String, type: String } ]
}
Let's say I have a mongodb bulk operation, containing two operations:
A) for a list of given documents, remove all…
dstibbe
- 1,589
- 18
- 33
0
votes
1 answer
SQL Server Complex Add/Update merge million rows daily
I have 7 reports which are downloaded daily at late night.
These reports can be downloaded in csv/xml. I am downloading them csv format as they are memory efficient.
This process runs in background and is managed by hangfire.
After they are…
Khalil
- 1,047
- 4
- 17
- 34
0
votes
2 answers
How do I transfer files from subfolders to folders two levels up while keeping the files in the main folder and deleting the subfolders?
I need a batch file to move the files from this:
F:\123456\V000\Chaptered\File1
F:\123456\V000\Chaptered\File2
F:\123457\V000\Chaptered\File1
F:\123457\V000\Chaptered\File2
to…
0
votes
1 answer
How to backfill column in Postgres without locking the table?
I recently added a new column to my 40+ million row Postgres table (v9.6) with
ALTER TABLE queries
ADD COLUMN ml_partition_source UUID;
Then in another transaction I executed
ALTER TABLE queries
ALTER COLUMN ml_partition_source
SET…
Paymahn Moghadasian
- 9,301
- 13
- 56
- 94
0
votes
1 answer
bulk is replacing other fields even with upsert declared
Why the bulk y replacing other fields if upsert is declare?
What am i doing wrong?
original collection:
{
_id:'xx',
a:1
}
process
var bulk = collection.initializeUnorderedBulkOp();
users.forEach(x => {
bulk
…
Fraga
- 1,361
- 2
- 15
- 47
0
votes
2 answers
Optimize bulk update with parallelization
I have a procedure used to scramble transactional data moving live data to a test environment. The table in question holds approx. 100 million rows spread across 50 partitions. A new partition is added every month. As the volume increase, procedure…
MrM
- 389
- 1
- 8
- 23