DLP data scan from bigquery table showing start byte as null

Question

I have scanned a Bigquery table from Google DLP Console. The scan results are saved back into a big query table. DLP has identified sensitive information, but the start byte is shown as null, can anyone help me understand why?

The source data looks as follows:

2,james@example.org  ,858-333-0333,333-33-3333,8
3,mallory@example.org,858-222-0222,222-22-2222,8
4,maria@example.org  ,858-444-0444,444-44-4444,1 

------------------------------

If I put the same data in Cloud storage bucket and then perform a scan using DLP, I get the start and end bytes for the sensitive data

You mean that you don't get the "null" column in your data output when analysing data stored in a bucket - right ? Can you post sanitised version of the file you have in the bucket ? — Wojtek_B, Oct 08 '20 at 13:02
Yes @Wojtek_B, that is right. When analysing data stored in a bucket, the output looks good. It gives the start bytes as not null values. Sure, here is the data that is stored in the storage bukcket: `1,mallory@example.org,858-222-0222,222-22-2222,5 2,james@example.org ,858-333-0333,333-33-3333,8 3,mallory@example.org,858-222-0222,222-22-2222,8 4,maria@example.org ,858-444-0444,444-44-4444,1` — Kuwali, Oct 08 '20 at 14:10

score 3 · Answer 1 · edited Oct 12 '20 at 19:27

3

Thanks folks, the product team is investigating. What's happening is that "0" is mapping to null "by accident" due to a proto to BQ schema conversion bug on our end. We'll address this.

edited Oct 12 '20 at 19:27

Mike DaCosta

1,340
1
10
10

answered Oct 12 '20 at 17:15

Jordanna Chord

950
5
12

score 1 · Accepted Answer · answered Oct 09 '20 at 13:59

Unfortunatelly this looks like a bug.

I was able to reproduce your issue completely; I fallowed these steps:

screated a source csv file:

1,mail1@test.com,858-333-0333,333-33-3333,8
2,epaweda-8101@yopmail.com,858-333-0334,333-33-3334,3
3,petersko@live.com,858-333-0335,333-33-3335,5
4,danneng@gmail.com,858-333-0336,333-33-3336,1
5,chance@icloud.com,858-333-0337,333-33-3337,4

imported it to a BQ table - it looks like this:
DLP'ed it and got the same result with null column:

In my opinion this is a bug (certainly looks like it) so my recommendation would be to go to Google's Issuetracker and report it here (with as much details as possible) and wait for an answer.

Thank you so much for your analysis @Wojtek_B. This really helped me ! — Kuwali, Oct 12 '20 at 05:56
Thanks @Kuwali for the confirmation - an "accept" would be very much appreciated :) — Wojtek_B, Oct 12 '20 at 07:23

DLP data scan from bigquery table showing start byte as null

2 Answers2