Stack Overflow
Stack Overflow

Questions tagged [chunking]

1. In memory management, chunking refers to strategies for improving performance by aggregating related memory-allocation requests. 2. In HTTP message transmission, it refers to a facility that allows inconveniently large messages to be broken into conveniently-sized smaller "chunks." 3. In parallel computing, it refers to the amount of data to assign to each task.

369 questions
9
votes
2 answers

What is the default chunker for NLTK toolkit in Python?

I am using their default POS tagging and default tokenization..and it seems sufficient. I'd like their default chunker too. I am reading the NLTK toolkit book, but it does not seem like they have a default chunker?
TIMEX
  • 259,804
  • 351
  • 777
  • 1,080
9
votes
1 answer

How to handle file upload chunks on the server (Plupload/Spring MVC)?

On our client, it is possible to be uploading large files. We want to use chunking to reduce the size of the requests. We are using Plupload, so it's easy to send up the files in chunks. However, I'm not sure how to handle the chunks. We are using…
dnc253
  • 39,967
  • 41
  • 141
  • 157
9
votes
2 answers

Memory usage serializing chunked byte arrays with Protobuf-net

In our application we have some data structures which amongst other things contain a chunked list of bytes (currently exposed as a List). We chunk bytes up because if we allow the byte arrays to be put on the large object heap then over…
James Thurley
  • 2,650
  • 26
  • 38
8
votes
1 answer

Webpack chunking. No content appearing - chunks not loaded

It's been a day I'm trying to solve this annoying but, I'm sure, simple issue. I am trying to divide my bundle.js into chunks to optimize website loading time. Here is my webpack.config file : module.exports = { devServer: { historyApiFallback:…
Louis
  • 418
  • 6
  • 22
8
votes
1 answer

Python: Chunking others than noun phrases (e.g. prepositional) using Spacy, etc

Since I was told Spacy was such a powerful Python module for natural speech processing, I am now desperately looking for a way to group words together to more than noun phrases, most importantly, prepositional phrases. I doubt there is a Spacy…
Malte Ge
  • 153
  • 3
  • 12
8
votes
1 answer

Using Pandas how do I deduplicate a file being read in chunks?

I have a large fixed width file being read into pandas in chunks of 10000 lines. This works great for everything except removing duplicates from the data because the duplicates can obviously be in different chunks. The file is being read in chunks…
Gregory Arenius
  • 2,904
  • 5
  • 26
  • 47
8
votes
1 answer

download file client-side chunk by chunk

I'm using WebRTC to send a file to a connected peer, and I'm sending the file in chunks. However, I'm having trouble figuring out how to get the peer to save/download the file as it is streaming in, chunk by chunk. All the examples I've found online…
Hristo
  • 45,559
  • 65
  • 163
  • 230
8
votes
1 answer

Compression performance related to chunk size in hdf5 files

I would like to ask a question about the performance of compression which is related to chunk size of hdf5 files. I have 2 hdf5 files on hand, which have the following properties. They both only contain one dataset, called "data". File A's…
C.T.
  • 95
  • 1
  • 3
7
votes
1 answer

Creating relations in sentence using chunk tags (not NER) with NLTK | NLP

I am trying to create custom chunk tags and to extract relations from them. Following is the code that takes me to the cascaded chunk tree. grammar = r""" NPH: {+} # Chunk sequences of DT, JJ, NN PPH: {} …
Rohan
  • 3,296
  • 2
  • 32
  • 35
7
votes
1 answer

How to terminate Eloquent chunking from closure

Using Eloquent, how can I terminate chunking based on a condition within the chunk function's closure? I've tried returning, but that seems to terminate only the current chunk and not all chunking. At this point, I want to stop retrieving records…
Brian
  • 432
  • 1
  • 5
  • 12
7
votes
2 answers

Chunk reading in Spring Batch - not only chunk writing

My assumption In my understanding "chunk oriented processing" in Spring Batch helps me to efficiently process multiple items in a single transaction. This includes efficient use of interfaces from external systems. As external communication includes…
Peter Wippermann
  • 4,125
  • 5
  • 35
  • 48
6
votes
13 answers

F# array_chunk for Sequence

I'm having some trouble making a sequence. Basically I need to chop a sequence into a sequence of arrays. Seq.windowed almost does it but I don't want duplicate elements. I can get what I want by reading everything into an array first but I'd…
gradbot
  • 13,732
  • 5
  • 36
  • 69
6
votes
1 answer

Pairwise Distance with Large NumPy Arrays (Chunking?)

Problem: I have a vector that is approximately [350000, 1] and I wish to calculate the pair wise distance. This results in a [350000, 350000] matrix of integer datatype that does not fit into RAM. I eventually want to end up with a boolean (which…
Daniel J
  • 73
  • 5
6
votes
1 answer

Use Scala Iterator to break up large stream (from string) into chunks using a RegEx match, and then operate on those chunks?

I'm currently using a not-very-Scala-like approach to parse large Unix mailbox files. I'm still learning the language and would like to challenge myself to find a better way, however, I do not believe I have a solid grasp on just what can be done…
foxtrotuniform6969
  • 3,527
  • 7
  • 28
  • 54
6
votes
1 answer

How to concatenate chunked file uploads from Dropzone.js with PHP?

I'm using Dropzone.js to take files of various types (including images and non-images, like a PDF), and upload them in 1mb chunks to our server. I'm then attempting to concatenate the files with PHP and later upload them to our company's FileMaker…
Grant Noe
  • 985
  • 8
  • 29
1 2
3
…
24 25