Stack Overflow
Stack Overflow

Questions tagged [chunking]

1. In memory management, chunking refers to strategies for improving performance by aggregating related memory-allocation requests. 2. In HTTP message transmission, it refers to a facility that allows inconveniently large messages to be broken into conveniently-sized smaller "chunks." 3. In parallel computing, it refers to the amount of data to assign to each task.

369 questions
14
votes
3 answers

What is a chunker in Natural Language Processing?

Does anyone know what is a chunker in the context of text processing and what is it's usage?
Hossein
  • 40,161
  • 57
  • 141
  • 175
14
votes
3 answers

split file on Nth occurrence of delimiter

Is there a one-liner to split a text file into pieces / chunks after every Nth occurrence of a delimiter? example: the delimiter below is "+" entry 1 some more + entry 2 some more even more + entry 3 some more + entry 4 some more + ... There are…
cmo
  • 3,762
  • 4
  • 36
  • 64
13
votes
4 answers

NLTK Chunking and walking the results tree

I'm using NLTK RegexpParser to extract noungroups and verbgroups from tagged tokens. How do I walk the resulting tree to find only the chunks that are NP or V groups? from nltk.chunk import RegexpParser grammar = ''' NP: {
?**} V:…
Vincent Theeten
  • 251
  • 1
  • 3
  • 7
12
votes
4 answers

WCF Chunking / Streaming

I'm using WCF and want to upload a large file from the client to the server. I have investigated and decided to follow the chunking approach outlined at http://msdn.microsoft.com/en-us/library/aa717050.aspx However, this approach (just like…
Mark Robinson
11
votes
1 answer

How can I upload large files by chunk, pieces?

I have got a little file sharing webpage. It's free to use it. I would like to upload files between 0mb and 1GB. I'm searching in Google since two days, but I can't find anything what I needed... My webpage: http://boxy.tigyisolutions.hu However I…
janos
  • 247
  • 2
  • 4
  • 11
10
votes
3 answers

For Google App Engine (java), how do I set and use chunk size in FetchOptions?

Im running a query and it is currently returning 1400 results and because of this I am getting the following warning in the log file: com.google.appengine.api.datastore.QueryResultsSourceImpl logChunkSizeWarning: This query does not have a chunk…
johnvdenley
  • 739
  • 1
  • 7
  • 16
10
votes
1 answer

How to download a big file directly to the disk, without storing it in RAM of a server and browser?

I want to implement a big file downloading (approx. 10-1024 Mb) from the same server (without external cloud file storage, aka on-premises) where my app runs using Node.js and Express.js. I figured out how to do that by converting the entire file…
Mike
  • 14,010
  • 29
  • 101
  • 161
10
votes
3 answers

Designing a Regex to find any Noun Phrase

I'm trying to build a chunker (or shallow parser) using regular expressions (and without NLTK), but can't come up with a regular expression that does what I want it to do . Here's my immediate goal: find all noun phrases in a natural language…
user3609038
  • 113
  • 1
  • 6
9
votes
1 answer

File uploads; How to utilize "chunking"?

I am (still) attempting to upload large files <200mb via a html form using php. During my research into this I have come across the term "chunking", I understand that this process can break the file into handy sizes such as 5mb and reassemble them…
Username_null
  • 1,249
  • 2
  • 21
  • 29
9
votes
3 answers

Handling large SQL select queries / Read sql data in chunks

I'm using .Net 4.0 and SQL server 2008 R2. I'm running a big SQL select query which returns millions of results and takes up a long time to fully run. Does anyone know how can I read only some of the results returned by the query without having to…
Omri
  • 1,058
  • 3
  • 14
  • 26
9
votes
1 answer

Trouble with RTMP ingest chunk stream

I am trying to build my own client RTMP library for an app that I am working on. So far everything has gone pretty successfully in that I am able to connect to the RTMP server negotiate the handshake and then send all the necessary packets…
Charlie
  • 222
  • 3
  • 20
9
votes
3 answers

Why would gnu parallel chunking improve gzip's compression size?

File under: "Unexpected Efficiency Dept." The first 90 million numbers take up about 761MB, as output by: seq 90000000 According to man parallel, it can speed up gzip's archiving big files by chopping the input up, and using different CPUs to…
agc
  • 7,973
  • 2
  • 29
  • 50
9
votes
1 answer

Relation extraction via chunking using NLTK

I am trying to figure out how to use NLTK's cascading chunker as per Chapter 7 of the NLTK book. Unfortunately, I'm running into a few issues when performing non-trivial chunking measures. Let's start with this phrase: "adventure movies between 2000…
grill
  • 1,160
  • 1
  • 11
  • 24
9
votes
1 answer

Rechunk a conduit into larger chunks using combinators

I am trying to construct a Conduit that receives as input ByteStrings (of around 1kb per chunk in size) and produces as output concatenated ByteStrings of 512kb chunks. This seems like it should be simple to do, but I'm having a lot of trouble,…
Rehno Lindeque
  • 4,236
  • 2
  • 23
  • 31
9
votes
2 answers

Laravel chunking not reducing PHP memory usage

So I've been trying my hands on laravel's chunking in Eloquent but I've run into a problem. Consider the following code (a much more simplified version of my problem): $data = DB::connection('mydb')->table('bigdata') ->chunk(200, function($data) { …
tiffanyhwang
  • 1,413
  • 4
  • 18
  • 26
1
2
3
…
24 25