4

I have Cloud Foundry and php app with mod-sec.

The app receives from the browser a json POST. The post contains several images coded in base 64 and Apace cut this in some lines:

2020-05-01T13:49:31.69+0200 [APP/PROC/WEB/0] OUT 11:49:31 httpd modsec body":["{\"image1\":\"data:image/png;base64, ...
2020-05-01T13:49:31.69+0200 [APP/PROC/WEB/0] OUT tTHNrVze+6IgS/ftH89uE2lw

Is there a way to concatenate these logs?

I wish to have:

2020-05-01T13:49:31.69+0200 [APP/PROC/WEB/0] OUT 11:49:31 httpd modsec body":["{\"image1\":\"data:image/png;base64, ...  tTHNrVze+6IgS/ftH89uE2lw\"}
snorlax
  • 157
  • 2
  • 3
  • 10
yalabef21
  • 151
  • 9

3 Answers3

2

I don't think this is an issue with Apache Web Server or mod_sec. Cloud Foundry will read logs from applications running on it, logs that are written to STDOUT/STDERR. This is the way you should be logging if you have an app running on Cloud Foundry, and if you're deploying a PHP app to Cloud Foundry that is how the PHP buildpack will configure PHP and the Apache Web Server to host your PHP application.

You cannot simply log to a file because the file system for your application running on Cloud Foundry is ephemeral. It will work during happy times, but when something fails and your app crashes it will not work. The file system, including the logs you've written to it, will be gone when the app crashes and that makes troubleshooting very difficult.

In regards to the behavior you're seeing, the Cloud Foundry logging system has size limits for a single line log entry. If you try to write a single line log entry that goes beyond the limit, the log system will automatically split the line and you'll end up with two log lines. I suspect that is what's happening here.

I did a little looking but couldn't find a documented max value for the length of a line before it'll be split. The best I could find was this discussion about the setting which provides some history about it. It doesn't seem to offer a clear explanation of how things are set up though.

Since I couldn't find an officially documented value, I wanted to test and validate the length. To do this, I deployed a test app to Pivotal Web Services (runs the latest version of OSS Cloud Foundry) and wrote some long log lines (if you'd like to see the test code, just let me know). The max I was able to get before the line is split was 61441 bytes (I repeated the a character in my test, if you have multi-byte characters like 丂 it will split sooner). This matched the older behavior mentioned in the discussion link above. It is possible your mileage will vary, depending on the version of CF you're using and the way your platform is configured.

Regardless of the exact value for the limit, there will always be some limit. You can either try to reconstitute your log entries, i.e. stitch them back together, after the fact, or you can store the information somewhere else. Use a service like a database or send the log entries directly to syslog.

Syslog is a good option, but HTTPD's built-in support for that requires syslogd to be running locally, which it's not inside a Cloud Foundry container. It's probably easier to use the piped logging feature and use an external program to send logs out. Here's an example of doing that.

Daniel Mikusa
  • 13,716
  • 1
  • 22
  • 28
1

Do not rely on the logs that you find printed in the terminals, sometimes they are nice because they have colors. However, the best and most complete choice is to rely on log files, because they shouldn't have abbreviations (hopefully).

As for Apache there are two type of apache httpd server log files:

  1. Error Logs
  2. Access Logs

Depending on the OS you are using, you may find the log file in different locations.

To find exact apache log file location, you can use grep command:

grep ErrorLog /usr/local/etc/apache22/httpd.conf
grep ErrorLog /etc/apache2/apache2.conf
grep ErrorLog /etc/httpd/conf/httpd.conf

Example output could be: ErrorLog "/var/log/httpd-error.log".

The same for Access Log:

grep CustomLog /usr/local/etc/apache22/httpd.conf
grep CustomLog /etc/apache2/apache2.conf
grep CustomLog /etc/httpd/conf/httpd.conf

Another possible solution would be to print the base-64 object directly to a file.

shogitai
  • 1,823
  • 1
  • 23
  • 50
  • The problem is that all the files are printed on the terminal in real-time and also in the files logs I have this problem :( – yalabef21 May 01 '20 at 12:40
  • Normally you would be right, but applications running on Cloud Foundry write all of their logs to STDOUT/STDERR. This information is the captured by the logging system and made available to users. The reason is because containers in CF are ephemeral and so if your app crashes, your log files would go away and you'd have no idea why it crashed. – Daniel Mikusa May 02 '20 at 17:39
  • It could be redirecting default STDOUT/STDERR to a file a solution? – shogitai May 03 '20 at 11:05
  • @kitsune - I would recommend logging to a file. See my answer for details on why & potential solutions. – Daniel Mikusa May 04 '20 at 15:22
1

One possibility would be to look at the Apache ErrorLogFormat directive. This allows you streamline the information in each Apache log, so you could create a format that would give you a shorter width (less verbose), which might help on your screen.

I know of no way to concatenate logs within Apache. It is simply listening to the PHP error_log command, so the PHP code could bundle it up and only call error_log once.

Katie
  • 2,594
  • 3
  • 23
  • 31
  • The log is created by Apache using mod-sec. It has not generated by PHP code. ``` SecAuditLog "|/bin/sed -u -e 's#^# modsec #" SecAuditLogFormat JSON ``` – yalabef21 May 01 '20 at 18:54
  • I see, I have not a solution for it. I ask me if there is a solution. – snorlax May 01 '20 at 19:16