1. In memory management, chunking refers to strategies for improving performance by aggregating related memory-allocation requests. 2. In HTTP message transmission, it refers to a facility that allows inconveniently large messages to be broken into conveniently-sized smaller "chunks." 3. In parallel computing, it refers to the amount of data to assign to each task.
Questions tagged [chunking]
369 questions
0
votes
1 answer
Padding is invalid and cannot be removed exception and length of the data to decrypt is invalid. Rijndael
My requirement is like below.
Step 1) I want to write 101 MB data in local file system by encrypting it in chunks of 10-10 MB.
Step 2) While decrypting, I want to fetch 10-10 MB data from file and want to decrypt and want to pass 10-10 MB…
0
votes
1 answer
Uploading files in chunks with Fine Uploader, corrupt when merged back together
I'm using fineuploader (https://fineuploader.com) with concurrent chunking to upload files to the server. For small files its working great but for large files I'm seeing an issue where the merged file ends up as corrupt.
E.g I tried a 2.7GB Zip…
Ben
- 11
- 4
0
votes
0 answers
Reading large HDF using with Fortran
I have some HDF data that has been created with PyTables. This data is very large, an array of 3973850000 x 8 double precision values, but with PyTables compression this can easily be stored.
I want to access this data using Fortran. I do,
PROGRAM…
user1887919
- 829
- 2
- 9
- 24
0
votes
1 answer
How does dataframe store large amount of data in memory and manipulate?
Suppose I have a large amount of data that I am loading into a dataframe by chunks;?
For eg : I have a table which is more than 40 Gb and selecting 3 columns may be around 2 - 3 gb suppose and records are 10 million (count of rows)
c =…
Viv
- 1,474
- 5
- 28
- 47
0
votes
1 answer
SourceNode.fromSourceWithMap is not a function in mocha-env-loader
I am trying to integrate Karma into React with Universal Routing. I removed the common chunk plugin because of an error with using common chunks with karma webpack
https://github.com/webpack-contrib/karma-webpack/issues/24#issuecomment-257613167
My…
Jack Shultz
- 2,031
- 2
- 30
- 53
0
votes
1 answer
Chunking parenthesis with NLTK
I am working with NLTK and I am trying to chunk (AIM 20-40-60) from the following text:
text = for more information refer to the Business Reporting Policy (AIM 20-40-60)
Currently I am using the following chunk pattern grammar =…
LoniF
- 97
- 1
- 1
- 7
0
votes
1 answer
Using chunk in json, requests to get large data into python
I am trying to get a large data into python using API. But I am not being able to get the entire data. The request is allowing only first 1000 lines to be retrieved.
r =…
amen
- 45
- 12
0
votes
1 answer
How to train a chunker on CONLL 2000 Corpora in Python?
I am new to Python, so I don't much about it. I am using nltk to tag Parts of speech in a sentence. I wan to chunk them afterwards. I am done with POS tagging. I don't know how to train chunker on CONLL 2000 corpora.
ALi Hamza
- 17
- 6
0
votes
0 answers
How to convert a csv to IOB format in python
I have a csv file which I would like to convert to IOB format so that I can train it using crfsuite. I am using python and I am new to it. Here is part of the data of the csv file
data of the csv file
this is how I want it to be..I think this is the…
0
votes
1 answer
How to output relationship between entities to if-then diagrams?
I am trying to extract chunk patterns using NLTK and I have 2 groups of patterns.One of them is input and the lead to the second group.Example:
with an example like: you may sell the house on a single…
Alma
- 97
- 1
- 12
0
votes
1 answer
How to set the "chunk size" of read lines from file read with Python subprocess.Popen() or open()?
I have a fairly large text file which I would like to run in chunks. In order to do this with the subprocess library, one would execute following shell command:
"cat hugefile.log"
with the code:
import subprocess
task = subprocess.Popen("cat…
ShanZhengYang
- 16,511
- 49
- 132
- 234
0
votes
3 answers
SQL Server: Why isn't this logic working when Chunking on inserts?
Fellow Techies--
I've got an endless loop condition happening here. Why is @@rowcount never getting set back to 0? I must not be understanding what @@rowcount really does--or I am setting the value in the wrong place. I think the value should be…
plditallo
- 701
- 2
- 13
- 31
0
votes
1 answer
Optimizing upload: request sent
I'm working on an async uploader in JavaScript intended to send multi-gigabyte files to our server efficiently. It uses JavaScript's FileReader to slice the files 5MB chunks at a time, and send 5 chunks concurrently.
Seems that the bottle neck is…
rodrigo-silveira
- 12,607
- 11
- 69
- 123
0
votes
2 answers
Is the callback from WCF to ASP.NET possible?
I have a Web service hosted in IIS in different box and WCF service hosted in Windows service, in a different box.(N-tier approach). Here, Web service is a client for WCF service. The request to upload the file comes to the IIS hosted Web service,…
MS_Windows
- 1
- 2
0
votes
0 answers
PHP - Convert binary chunks to a video file - feasible?
I am creating an application which requires slicing a video file (mp4 format) into chunks. Our server caps the upload_max_filesize at 2MB, but we have files that are hundreds of MBs in size which require uploading. So far I slice the file (into 1MB…
The One and Only ChemistryBlob
- 7,864
- 10
- 36
- 69