17

The suggestion to encrypt log files as a means of protecting the personal data that might be contained in them is widespread.

What I've not seen is a good reference implementation, which is surprising given how many companies will need this.

In our particular case, we want to use public key encryption so that the files can not be read on the (weakly protected) system that generates them, and must be sent back to head office where we can look at them.

The best suggestion I've seen so far is "use log4net but write your own appender using the RFC 3852 streaming implementation from BouncyCastle". Does anyone have an advance on that?

pjc50
  • 1,856
  • 16
  • 18
  • Personal data? Log files? Log files aren't meant to contain any personal data, log files are for debugging, you should never write anything private to a log file. – Gusman Mar 15 '18 at 14:52
  • 4
    It's extremely useful to log the name or userid of the person performing an action. Ours also includes all input button presses, which includes any data entered albeit not in a convenient form. – pjc50 Mar 15 '18 at 14:56
  • 1
    (Would be nice to know why this is getting close and negative votes!) – pjc50 Mar 15 '18 at 15:00
  • Downvote and closes are because this is too broad for S.O. About the logging, userId yes, that's not private at all, username no, don't log it. And never log anything what can contain private data, that's a bomb waiting to explode. – Gusman Mar 15 '18 at 15:05
  • 1
    There are tons of more general questions still open such as "how do I do logging in c#" https://stackoverflow.com/questions/5057567/how-do-i-do-logging-in-c – pjc50 Mar 15 '18 at 15:06
  • You can't compare a user asking how to write a file to your question... – Gusman Mar 15 '18 at 15:46
  • 5
    Oh dear, looks like Wikipedia-style bureaucracy games are taking over. This is a specific question, with a precise example. – Reuben Thomas Mar 15 '18 at 22:59
  • 8
    This is an excellent question. Logs often, be it unintentionally, contain PII data (ex: "could not parse line: 'Mr Anderson, Primrose Av. 81 &*#'"). Sometimes logs are not just written to disk but also mailed to the admin, so they proliferate through the organisation. – Ivana Mar 16 '18 at 10:06
  • 2
    @Gusman I don't agree. If you think that the correct answer is "you should never log PII data" then make that your answer, not a random comment. How to do encrypted logging sounds like a really great question, especially since GDPR is so important to so many people (and encryption alone won't give you GDPR-compliance). – Adam Batkin Mar 18 '18 at 13:53
  • @Gusman Even if a log contains no personal identification, I do not think it can be ruled out that a such a log could, in some circumstances, be of advantage to an attacker who already has partial information. In that case, the principle of defense-in-depth suggests that logs of systems with secrets to hide should be encrypted. – sdenham Mar 18 '18 at 14:22
  • @Ivana Excellent for [softwareengineering.SE](https://softwareengineering.stackexchange.com/), not stackoverflow – Izkata Mar 18 '18 at 15:56
  • 1
    @sdenham If an attacker access to your local logs, you have more things to worry about than the logs... – Gusman Mar 18 '18 at 16:22
  • 1
    @AdamBatkin Logs, as we understand them should contain no sensible data at all, they must be tools for debugging purposes, following that rule you're safe for GDPR compliance, if you must store sensible data another mechanism than regular logs should be used. – Gusman Mar 18 '18 at 16:23
  • 1
    PII, PID & PCI, or any form of identifiable information, shouldn't really be logged as that opens up the possibility of that data being displayed on monitoring tools such as Kibana. What would be better to do is log a Guid that would identify the search, and then store the identifiable information in your database (search audit) with encryption at rest. That way, you can apply data retention and comply with GDPR regulation. – ColinM Mar 18 '18 at 17:44
  • 1
    See [Log4NetMessageEncryptor](https://github.com/ArtisanCode/Log4NetMessageEncryptor). There is a corresponding Nuget package as well. You can also take this as your starting point if you wanted a different type of encryption. Asking for a complete implementation to your specification is considered out of scope on [so]. See also [ask]. If you get stuck with your implementation and have a *specific* question or problem then please come back and ask. – Igor Mar 18 '18 at 18:03
  • 1
    @Gusman The issue (and the point of defense-in-depth in general) is whether lack of encryption could make a bad situation worse. You could substitute 'database' for 'logs' in your reply, and you would have what looks like an endorsement of Equifax's security. – sdenham Mar 19 '18 at 12:29

3 Answers3

4

Technically, encrypting your log messages should be pretty easy. Using something like Serilog you could simply create a custom sink.

Just blind encrypting the whole log is probably going to limit the usefulness of the logs though. If you're centralizing your logging using something like ELK then you won't be able to search based on any field/part of your logs that you encrypt (for example, if you encrypt the machine name then you don't even know where the logs come from!).

If the kind of information that you're dealing with genuinely is personally identifiable information covered by GDPR then maybe you just have to suck that up - but I'd make an effort to encrypt only sensitive information from your logs rather than just blanket encrypting everything... that would require a more sophisticated sink but it will make your log data way less crippled.

James Crosswell
  • 648
  • 4
  • 13
3

I agree with some of the commentators; personal data should not be a part of the log files. GDPR is not about the encryption - if you just encrypt personal data that does not mean that you're GDPR compliant. What will happen with the personal data in your log files when you receive "forget me" (Right to erasure) request from the individual? Or "change my data" (Right to rectification)?

However, if you need to log personal data, maybe the option can be to hash the information and store hashed version in the logs. In that case, you'll be able to find the specific data in the logs, by calculating the hash from the search string.

Related to a public key encryption part of your question, take a look: https://aws.amazon.com/kms or https://azure.microsoft.com/en-us/services/key-vault/

Nino
  • 51
  • 1
  • 2
  • I think this answers the question better, not by directly answering the encryption question - but by getting the point across of what GDPR compliance means. The text/log files would have to be decrypted on a scheduled basis and data be removed in order to be compliant. How fun would it be to respond to an FSAR request when working with plain text log files with no indexing. – ColinM Mar 26 '18 at 10:26
0

If you're on a hosted server, follow James's answer, make a custom sink, and have it log the file in a way you control it.

I prefer to use a system-wide type of solution and let the operating system take care of access control; segregation of responsibility is a thing that can simplify things.

Using a group policy, you can instruct windows to protect data where only authorized users can read the data consistently in your organization.

Have a look at the following: https://techcommunity.microsoft.com/t5/windows-server-essentials-and/help-secure-your-business-information-using-encrypting-file/ba-p/397386

Contact your IT support and as them to protect the GDPR folders for you.

Walter Verhoeven
  • 3,867
  • 27
  • 36