8

When we check the register usage by using xptxas we see something like this:

ptxas info : Used 63 registers, 244 bytes cmem[0], 51220 bytes cmem[2], 24 bytes cmem[14], 20 bytes cmem[16]

I wonder if currently there is any documentation that clearly explains cmem[x]. What is the point of separating constant memory into multiple banks, how many banks are there in total, and what are other banks other than 0, 2, 14, 16 used for?

as a side note, @njuffa (special thanks to you) previously explained on nvidia's forum what is bank 0,2,14,16:

Used constant memory is partitioned in constant program ‘variables’ (bank 1), plus compiler generated constants (bank 14).

cmem[0]:kernel arguments

cmem[2]:user defined constant objects

cmem[16]:compiler generated constants (some of which may correspond to literal constants in the source code)

einpoklum
  • 118,144
  • 57
  • 340
  • 684
biubiuty
  • 493
  • 6
  • 17
  • 5
    You are welcome. I think I mentioned in my post in the NVIDIA forums that the number of constant banks differs between GPU architectures, as does the bank assignment. In other words, these are implementation detail that programmers should not worry about as they are not part of the programming model. One reason for using multiple banks is to minimize the potential for conflicting uses of constant memory, in particular keeping as much of the programmer-visible constant bank available for user code. – njuffa Sep 05 '12 at 23:29
  • @njuffa I think this (along with the quotes from your forums post above) should be converted to an answer. :) – harrism Sep 06 '12 at 03:18
  • @njuffa I have a follow-up question. The programming guide says _The arguments to the execution configuration are evaluated before the actual function arguments and like the function arguments, are currently passed via shared memory to the device._ My understanding is at compile-time, the arguments are copied to cmem[0], but at run-time they are copied from cmem to smem right before a new block starts. Is that correct? – biubiuty Sep 06 '12 at 03:35
  • 3
    I am almost sure that the cited information applies to sm_1x devices only. I am seeking clarification from people more knowledgable about this. As I recall, for sm_1x devices a 16 byte block at the start of shared memory stored the launch configuration, followed by the kernel function arguments. For later chips shared memory does not come into play at all (leaving it to the programmer in its entirety): launch configuration is passed in special registers while kernel function arguments are stored in a constant bank. Implementation details programmers should not worry about. – njuffa Sep 06 '12 at 03:55
  • 3
    I confirmed that passing of launch configuration via shared memory applies to sm_1x only. Beyond that, I am told that sm_2x and sm_3x handle the passing of launch arguments differently. This highlights the fact that it is futile to try and keep up with constantly changing implementation details (same as for usage of register banks). @harrism: As I recall I reverse engineered constant bank assignments adhoc for a sample program, and I don't have that information anymore nor do I have access to my old forum post. I assume it was quoted accurately but no way to know for sure. – njuffa Sep 06 '12 at 04:14
  • I assume numbers 1 to 5 are used by the equivalent architecture. – kon psych May 29 '15 at 22:30

2 Answers2

6

The usage of GPU constant banks by CUDA is not officially documented to my knowledge. The number and usage of constant banks does differ between GPU generations. These are low-level implementation details that programmers do not have to worry about.

The usage of constants banks can be reversed engineered, if so desired, by looking at the machine code (SASS) generated for a given platform. In fact, this is how I came up with the information cited in the original question (this information came from an NVIDIA developer forum post of mine). As I recall, the information I gave there was based on adhoc reverse engineering specifically applied to Fermi-class devices, but I am unable to verify this at this time as the forums are inaccessible at the moment.

One reason for having multiple constant banks is to reserve the user visible constant memory for the use of CUDA programmers, while storing additional read-only information provided by hardware or tools in additional constant banks.

Note that the CUDA math library is provided as source files and the functions get inlined into user code, therefore constant memory usage of CUDA math library functions is included in the statistics for the user-visible constant memory.

njuffa
  • 23,970
  • 4
  • 78
  • 130
0

Please see "Miscellaneous NVCC Usage". They mention, that the constant bank allocation is profile-specific.

In the PTX guide, they say that apart from 64KB constant memory, they had 10 more banks for constant memory. The driver may allocate and initialize constant buffers in these regions and pass pointers to the buffers as kernel function parameters.

I guess, that profile given for nvcc will take care of what constants go into which memory. Anyway, we don't need to worry if each constant memory cmem[n] is less than 64KB, because each bank is of size 64KB and common to all threads in grid.

lxkarthi
  • 336
  • 4
  • 14