1

I would like to find a way for data (L1/L2) cache invalidation and flush in usermode on a Linux platform working with heterogenous non coherent caches (ARM A53 and ARM M7 cores). My problem lies on the A53 side, where an SMP Linux is running. M7 cores run a bare metal program on which I already implemented the flush / invalidation.

I know it is possible to write a kernel module for that but before doing this, I would like to know if there is any hidden API to do so? I am running on Linux 5.10.120. If not, what would be the most performant way to perform the flush / invalidation?

I used __builtin___clear_cache, however, I found out this was for instructions only. Also I would like to have a better granularity on the operation to avoid invalidating line when flushing if not needed. Also, the cacheflush function in asm/cachectl.h is not available (the header is not present for my CPU / target).

Disclaimer: This question was asked many times. Most answers try to be smart by telling "You don't need to do that...". Please refrain from answering something like this as the context in which I am asking this question requires data cache flush / invalidation.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
ØXmose
  • 13
  • 4

1 Answers1

0

If you are running 32-bit user-space software, then you are out of luck. You cannot access data cache maintenance ops from user-space.

(Updated) If you are running in Aarch64 64-bit mode, AND if the kernel enabled access (SCTLR_EL1.UCI and SCTLR_EL2.UCI set to 1), then you can access a few of the by-VA data cache maintenance instructions (DC CVAU, DC CIVAC, and DC CVAC).

I used __builtin___clear_cache, ... I would like to have a better granularity on the operation to avoid invalidating line when flushing if not needed.

Better granularity than what? builtin___clear_cache lets you specify start and end, so what more granularity do you need?

For the "by VA" instructions you can enable access to from user-space, you have no choice by to iterate and clean/invalidate line-by-line for every 64 byte chunk of your buffer ...

Depending on the amount of data your are transfering, it's often better to set up a non-cached DMA region to avoid the cost of the cache maintenance.

solidpixel
  • 10,688
  • 1
  • 20
  • 33
  • Thank you for the detailed answer, I will check this out! By better granularity I meant on the operation. I read that __builtin_clear_cache both flushed and invalidated the cache, while I was searching for invalidation only and flush only. I am searching for cache maintenance because my chunks of data to be invalidated / flushed range from 32B to up to 6MB so IMO caching still would be helpful to avoid multiple partial writes to the memory on memcopy operations. – ØXmose Dec 01 '22 at 09:19
  • The user-space cache instructions don't allow invalidate (it's a destructive operation), and IIRC Cortex-A53 invalidate operations also clean anyway. – solidpixel Dec 01 '22 at 19:18
  • You can still use the write-buffer for uncached write merging, so sequential accesses are still likely to be fast as long as the DMA region is allowing merging. – solidpixel Dec 01 '22 at 19:18
  • Thanks a lot for the advices! Unfortunately, we don't plan to use DMA regions as the communication between the coherent A53 would also be impacted (our allocation granularity forces us to allocate all the shared memory in the same area). We still want to keep communication between the A53 cached, and of course, without using cache maintenance. – ØXmose Dec 06 '22 at 08:30