Stack Overflow
Stack Overflow

Questions tagged [bulkloader]

167 questions
0
votes
1 answer

Spring Batch Perfomance is less compared to bufferedReader and JDBC Batch Inserts why?

I have a file with 80 String columns, with approx 2 Lacs record in it. I was evaluating technology to be used as in future we will be getting 2000 Lacs record file. I have evaluated 2 things Oracle 11 g Database Approach 1 - Spring…
Rahul Agrawal
  • 8,913
  • 18
  • 47
  • 59
0
votes
0 answers

Cannot Release Memory using BulkLoader AS3

I am currently using arthur-debert's BulkLoader in My Project, as I need to load a list of SWF and display them on screen. However, once I try to dispose the BulkLoader instance and reload another list, problem occurs: There is a lot of memory…
Jacky Lau
  • 665
  • 5
  • 21
0
votes
1 answer

Updating data with bulkloader

I'm using python script do upload some data to my app engine backend. Here is it's definition in bulkloader.yaml - kind: Subcategory connector: csv connector_options: encoding: utf-8 property_map: - property: __key__ …
Jacek Kwiecień
  • 12,397
  • 20
  • 85
  • 157
0
votes
1 answer

Hive load CSV: load part of columns (or column mapping)

I have CSV file with 20 columns but I need only three of them. Is is possible to load part of columns? Something like column mapping during the bulk load in MySQL.
Tioma
  • 2,120
  • 9
  • 35
  • 52
0
votes
1 answer

Upgrading from Cassandea 1.1.9 to 1.2.18

I'm trying to upgrade from a 3 node 1.1.9 cluster to a 6 node 1.2.18 cluster. Can sstableloader be used to stream from the existing cluster to the new cluster? If so, what that the suggested method? I keep getting the following when trying…
msd
  • 1
0
votes
2 answers

Where are the entities after uploading data?

I'm trying to import my csv file into the appengine datastore, and don't get any errors, but where are the entities? I could not see it in my appengine console in the datastore section. models.py: from google.appengine.ext import db class…
user3802804
  • 11
  • 3
0
votes
0 answers

Bulk uploading "search api" documents to appengine?

We would like to upload about 30k entities to the datastore in one go, while also creating documents from strings associated with these entities. This is to allow for partial search on strings which the datastore is not suited for. However, we…
Cygorger
  • 772
  • 7
  • 15
0
votes
2 answers

How to download all GAE datastore records?

I use GAE bulk loader to download datastore data - appcfg.py download_data --log_file=bulkloader.log --kind=MyKind --application=s~myappid --url=http://myappid.appspot.com/rmt_api --filename=data_downloaded.csv --db_filename=skip…
LA_
  • 19,823
  • 58
  • 172
  • 308
0
votes
0 answers

App Engine Bulkloader to Update Entities Instead of Replacing

I need to perform a nightly update in the datastore on a relatively large dataset (syncing a subset of corporate data with GAE). I've been using the bulkloader, and it does the job, but the write costs are really adding up. Since I'm specifying key…
Simon Phoenix
  • 195
  • 6
0
votes
1 answer

App Engine bulkloader transformation for datetime

I'm trying to load a datetime field with the following format. 2013-02-05T10:09:38-08:00 - property: event_time external_name: datetime import_transform: transform.import_date_time('%Y-%m-%dT%H:%M:%S%z') However, the transformer does not accept…
Greg
  • 2,559
  • 4
  • 28
  • 46
0
votes
1 answer

App Engine bulkloader - Loading Entities with parent keys?

There seems to be very little information regarding loading Entities with parent keys using the bulkloader for App Engine. I tried following the instructions that I found on here on StackOverflow... Uploading Entity with Parent Using Bulkloader But…
edcincy
  • 321
  • 2
  • 3
  • 14
0
votes
0 answers

Check for required columns in MySqlBulkLoader c#

Is there any way to validate the required columns when using MySqlBulkLoader ? I tried making the column in MySql table not nullable and no default. But still the values are getting loaded with no exception. Also, I wasnt able to find any…
wishy
  • 1,748
  • 2
  • 12
  • 14
0
votes
0 answers

"No implementation for kind" importing data to GAE

I am attempting to use the below loader class to import data from a bulkloader generated CSV: import datetime from google.appengine.ext import db from google.appengine.tools import bulkloader from market import Market class…
ljs.dev
  • 4,449
  • 3
  • 47
  • 80
0
votes
1 answer

GAE Python bulk uploader is giving error: "ImportError: No module named model"

I'm trying to use the bulk uploader in my development environment and I'm running into problems. Here is the error I'm getting: File "volume_loader.py", line 3, in import model ImportError: No module named model I'm in my application…
b-ryce
  • 5,752
  • 7
  • 49
  • 79
0
votes
0 answers

The type or namespace name 'MySqlBulkLoader' could not be found

I'm trying to use the MySQL .Net connector to upload csv to MySQL but there's no 'MySqlBulkLoader' object in the dll. I downloaded the binary from the MySQL site. using MySql.Data.MySqlClient; ... MySqlBulkLoader bl = new…
A.G.
  • 2,089
  • 3
  • 30
  • 52
1 2 3
…
11 12