How to calculate message digests in custom output stream?

Question

I would like to implement an OutputStream that can produce MessageDigests. Likewise, I already have an InputStream implementation of it here, which works fine and extends FilterInputStream.

The problem is this: if I'm extending FilterOutputStream, the checksums don't match. If I use FileOutputStream it works fine (although that is not the stream I'd like to be using, as I'd like it to be a bit more generic than that).

public class MultipleDigestOutputStream extends FilterOutputStream
{

    public static final String[] DEFAULT_ALGORITHMS = { EncryptionConstants.ALGORITHM_MD5,
                                                        EncryptionConstants.ALGORITHM_SHA1 };

    private Map<String, MessageDigest> digests = new LinkedHashMap<>();

    private File file;


    public MultipleDigestOutputStream(File file, OutputStream os)
            throws NoSuchAlgorithmException, FileNotFoundException
    {
        this(file, os, DEFAULT_ALGORITHMS);
    }

    public MultipleDigestOutputStream(File file, OutputStream os, String[] algorithms)
            throws NoSuchAlgorithmException, FileNotFoundException
    {
        // super(file); // If extending FileOutputStream
        super(os);

        this.file = file;

        for (String algorithm : algorithms)
        {
            addAlgorithm(algorithm);
        }
    }

    public void addAlgorithm(String algorithm)
            throws NoSuchAlgorithmException
    {
        MessageDigest digest = MessageDigest.getInstance(algorithm);

        digests.put(algorithm, digest);
    }

    public MessageDigest getMessageDigest(String algorithm)
    {
        return digests.get(algorithm);
    }

    public Map<String, MessageDigest> getDigests()
    {
        return digests;
    }

    public String getMessageDigestAsHexadecimalString(String algorithm)
    {
        return MessageDigestUtils.convertToHexadecimalString(getMessageDigest(algorithm));
    }

    public void setDigests(Map<String, MessageDigest> digests)
    {
        this.digests = digests;
    }


    @Override
    public void write(int b)
            throws IOException
    {
        super.write(b);

        System.out.println("write(int b)");

        for (Map.Entry entry : digests.entrySet())
        {
            int p = b & 0xFF;
            byte b1 = (byte) p;

            MessageDigest digest = (MessageDigest) entry.getValue();
            digest.update(b1);
        }
    }

    @Override
    public void write(byte[] b)
            throws IOException
    {
        super.write(b);

        for (Map.Entry entry : digests.entrySet())
        {
            MessageDigest digest = (MessageDigest) entry.getValue();
            digest.update(b);
        }
    }

    @Override
    public void write(byte[] b, int off, int len)
            throws IOException
    {
        super.write(b, off, len);

        for (Map.Entry entry : digests.entrySet())
        {
            MessageDigest digest = (MessageDigest) entry.getValue();
            digest.update(b, off, len);
        }
    }

    @Override
    public void close()
            throws IOException
    {
        super.close();
    }

}

My test case (the asserted checksums have been checked with md5sum and sha1sum):

public class MultipleDigestOutputStreamTest
{


    @Before
    public void setUp()
            throws Exception
    {
        File dir = new File("target/test-resources");
        if (!dir.exists())
        {
            //noinspection ResultOfMethodCallIgnored
            dir.mkdirs();
        }
    }

    @Test
    public void testWrite()
            throws IOException,
                   NoSuchAlgorithmException
    {
        String s = "This is a test.";

        File file = new File("target/test-resources/metadata.xml");

        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        MultipleDigestOutputStream mdos = new MultipleDigestOutputStream(file, baos);

        mdos.write(s.getBytes());
        mdos.flush();

        final String md5 = mdos.getMessageDigestAsHexadecimalString("MD5");
        final String sha1 = mdos.getMessageDigestAsHexadecimalString("SHA-1");

        Assert.assertEquals("Incorrect MD5 sum!", "120ea8a25e5d487bf68b5f7096440019", md5);
        Assert.assertEquals("Incorrect SHA-1 sum!", "afa6c8b3a2fae95785dc7d9685a57835d703ac88", sha1);

        System.out.println("MD5:  " + md5);
        System.out.println("SHA1: " + sha1);
    }

}

Could you please advise as to what could be the problem and how to fix it? Many thanks in advance!

Hannes · Answer 1 · 2015-04-07T18:17:14.810

If you are using java 7 or above, you can just use DigestOutputstream.

UPDATE

You can inplement the abstract MessageDigest class to wrap multiple MessageDigest instances.

SOME CODE

    import java.security.DigestException;
    import java.security.MessageDigest;
    import java.security.NoSuchAlgorithmException;


    public class DigestWrapper extends MessageDigest
    {
        private final MessageDigest md5;
        private final MessageDigest sha1;

        // some methods missing.
        // I just implemeted them throwing a RuntimeException.

        public DigestWrapper() throws NoSuchAlgorithmException
        {
            super(null);
            sha1 = MessageDigest.getInstance("sha-1");
            md5 = MessageDigest.getInstance("md5");
        }

        public byte[] getMD5Digest()
        {
            return md5.digest();
        }

        public byte[] getSHA1Digest()
        {
            return sha1.digest();
        }

        @Override
        public int digest(byte[] buf, int offset, int len) throws DigestException
        {
            md5.digest(buf, offset, len);
            sha1.digest(buf, offset, len);
            return 0;
        }

        @Override
        public byte[] digest(byte[] input)
        {
            md5.digest(input);
            sha1.digest(input);
            return input;
        }

        @Override
        public void reset()
        {
            md5.reset();
            sha1.reset();
        }

        @Override
        public void update(byte input)
        {
            md5.update(input);
            sha1.update(input);
        }

        @Override
        public void update(byte[] input, int offset, int len)
        {
            md5.update(input, offset, len);
            sha1.update(input, offset, len);
        }

        @Override
        public void update(byte[] input)
        {
            md5.update(input);
            sha1.update(input);
        }

    }

The problem with `DigestOutputStream` is that it only works with one digest at a time. — carlspring, Apr 07 '15 at 17:05
@carlspring You can - as a hack - wrap to Digest in one Digest wrapper. — Hannes, Apr 07 '15 at 18:18

score 1 · Answer 2 · answered Oct 14 '16 at 11:15

I have created a project on Github which contains my implementation of the MultipleDigestInputStream and MultipleDigestOutputStream here.

To check how the code can be used, have a look at the following tests:

Let me know, if there is enough interest and I can release it and publish it to Maven Central.

How to calculate message digests in custom output stream?

2 Answers2