1

I am using locust helm chart. I need to perform load testing by supplying a text files whose size is greater than 3MB. The configuration maps do not work such big files. I tried splitting my files into smaller chunks of less than 1MB. But I am still getting the error "requested entity too large: limit is 3145728" as I got before when the file was a single huge one. Is there any option available?

Pramavi
  • 11
  • 2

4 Answers4

0

Put the file somewhere accessible via the network and curl it down to a shared emptyDir volume using an initContainer. You can also use a ROX volume that you've populated manually if your hosting environment/provider offers those.

coderanger
  • 52,400
  • 4
  • 52
  • 75
0

You got this error due to a limitation for ConfigMap that is equal to 1Mb. According to the official ConfigMap documentation:

A ConfigMap is not designed to hold large chunks of data. The data stored in a ConfigMap cannot exceed 1 MiB. If you need to store settings that are larger than this limit, you may want to consider mounting a volume or use a separate database or file service.

At this point I can suggest you check @coderanger answer - that's the only available options for you.

  • To read and check

1)Configure a Pod to Use a Volume for Storage

2)Great SO answer Make large static data files available to kubernetes pods

Vit
  • 7,740
  • 15
  • 40
0

You may also place the file on a persistent volume and mount the volume to the pod. Also, you may also use a hostPath volume but your pod will then be scheduled to only that host.

Rakesh Gupta
  • 3,507
  • 3
  • 18
  • 24
0

You can use the PVC or use the Node path for storing the file.

PVC : https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes

If you are using the Host path method you have to make sure your pod each time get schedule on same Node or host.

For that you can also use : https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/

Or you can use the taints & toleration.

Harsh Manvar
  • 27,020
  • 6
  • 48
  • 102
  • 1
    How can you fill the PVC with a > 1MB file if your chart can't be more than 1MB? I'm pretty sure OP wanted (and I'm interested too) to include the file *as part of the deployment process*... Creating a PVC manually and uploading manually the data is a workaround... – GACy20 Jan 21 '22 at 15:58
  • If your application need those files as startup or so and you want clone those you can clone the disk. If you are on managed cloud pvc baked by pv and pv mean disk, you can clone and attach and use it. If you don’t want to follow that manual process you can use this: https://velero.io/ backup and restore. Take pvc snapshot and create new one anywhere. – Harsh Manvar Jan 21 '22 at 17:37