Stack Overflow
Stack Overflow

Questions tagged [zero-copy]

"Zero-copy" describes computer operations in which the CPU does not perform the task of copying data from one memory area to another.

Zero-copy describes computer operations in which the CPU does not perform the task of copying data from one memory area to another. This technique is used in various software elements (e.g.device drivers, file systems, and network protocol stacks) to increase the performance and more efficiently utilize system resources.

Network applications that use zero copy request that the kernel copy the data directly from the disk file to the socket, without going through the application. Zero copy greatly improves application performance and reduces the number of context switches between kernel and user mode.

References:

100 questions
2
votes
3 answers

how to refer part of an array?

Given an object byte[], when we want to operate with such object often we need pieces of it. In my particular example i get byte[] from wire where first 4 bytes describe lenght of the message then another 4 bytes the type of the message (an integer…
vach
  • 10,571
  • 12
  • 68
  • 106
2
votes
0 answers

What are the semantics of splice(2) when the source file changes after splicing?

I'm trying to figure out the semantics of splice(2) (man page here). Let's say I've got a regular file on disk, and I want to write a portion of its current contents into a pipe (actually into /dev/fuse on a system with the FUSE_SPLICE_WRITE…
jacobsa
  • 5,719
  • 1
  • 28
  • 60
2
votes
2 answers

is there transferFrom like functionality in Netty for zero copy?

I'm developing a Http server that processes fairly large size of payloads. Since Netty provide zero-copy, I thought of using zero-coping of the payload using Netty's zero-copy. But it seems Netty only provide transferTo(WritableByteChannel target,…
Sudheera
  • 1,727
  • 1
  • 12
  • 28
2
votes
1 answer

Alternatives to splice(2) with Unix Domain Sockets (UDS)

I'm trying to pass data from one UDS to another as quickly as possible in a C++ program, but UDS's don't support splice (as fd_in). Are there any alternative that are supported by UDS's that are as close to zero-copy as possible? This question was…
Brandon Johnson
  • 23
  • 4
2
votes
1 answer

Zero copy receive function

I am trying to implement zero copy mechanism i.e. a contrary function to the TransmitFile() API. But I don't know how to register buffers in kernel. Below are the details what I want to do to implement zero copy: We will receive a large data from…
Satya Kumar
  • 179
  • 1
  • 4
  • 19
2
votes
1 answer

How to cope with 320 million 272-byte UDP packets?

So, I have an incoming UDP stream composed of 272 byte packets at a data rate of about 5.12Gb/s (around 320e6 packets per second). This data is being sent by an FPGA-based custom board. The packet size is a limit of the digital design being run, so…
lessju
  • 148
  • 1
  • 12
1
vote
1 answer

Zero copy in sockets

I am writing a network application that will transfer data between a pair of sockets. Whenever some data is available in the source socket, it will transfer it to the destination socket and vice versa. But here I am looking for the most efficient…
Ajith C Narayanan
  • 564
  • 2
  • 21
1
vote
0 answers

Does pybind11 create a copy when passing bytes/string from python to C++?

I'm trying to use pybind11 to bind heavy computing in C++ with python as front end (unfortunately). I need to pass string/bytes around from python to C++ and back. After reading through the documentation, I thought that "without conversion" meaning…
npsable
  • 13
  • 3
1
vote
0 answers

Gstreamer zero copy dmabuf encoding

I want to encode a framebuffer I got via dmabuf into a video stream. I have a dmabuf file descriptor which contains the framebuffer. I got the filedescriptor from the intel i915 driver via ioctl VFIO_DEVICE_QUERY_GFX_PLANE. Now I want to encode it…
Xaterus
  • 21
  • 1
  • 3
1
vote
1 answer

How is splice() actually doing zero-copy in Linux?

I'm new to the concept of zero-copy but from what I understand, it is a way not to copy anything from kernel buffers to user buffers and pass data directly between the 2 kernel buffers. In this way, the CPU does not have to do 2 copies of data from…
Aritra Sur Roy
  • 182
  • 13
1
vote
1 answer

Netty using ByteBuffer from pooled direct ByteBuf

I try to understand few things about netty's ByteBuf and its relation to nio.ByteBuffer. the problem I try to solve is that my server has to return random portion of a file located on the server - this was implemented using nio.FileChannel and…
kamiseq
  • 593
  • 4
  • 17
1
vote
0 answers

High perf content server in java

Consider a very thin rest layer over S3 , the upload api has a file attachment ( multipart* MIME type ) . The service receives the call and forwards the bytes to S3 . what are my options to optimize performance here ?? I am thinking zero-copy…
redzedi
  • 1,957
  • 21
  • 31
1
vote
0 answers

Can memory map be used for an already allocated memory buffer?

I'd like to be able to share a pre-allocated memory between processes. Searching for an example to do it, I can only find a way to create a new file with shm_open, then use mmap and memcpy. This is a problem since the buffers are quite large and I…
miluz
  • 1,353
  • 3
  • 14
  • 22
1
vote
0 answers

Haskell SendFile zero-copy - Abort the transfert

Hi I want to use the following lib in Haskell for transferring files between nodes : https://hackage.haskell.org/package/sendfile-0.7.11.1/docs/Network-Socket-SendFile.html I want to be able to abort the sendFile if the file has been transferred by…
Nicolas Henin
  • 3,244
  • 2
  • 21
  • 42
1
vote
0 answers

Zero-copy in .NET Core

I have large file (usually 300 MB to 2 GB) which consists of smaller files. I want to unpack them to folder and I want to do that efficiently (zero-copy). I found, that .NET Framework have TransmitFile method for that using (StreamWriter writer =…
Matěj Pokorný
  • 16,977
  • 5
  • 39
  • 48
1 2 3
4
5 6 7