Unable to mock AWSClient, S3Object while reading a file from S3 bucket

Question

I am reading a file from S3 bucket as below:

class VersionServiceImpl implements VersionService {
VersionDto versiondto = new VersionDto();
    try {
        BasicAWSCredentials awsCreds = new BasicAWSCredentials(versionConfig.getAccessKeyId(), versionConfig.getSecretAccessKey());
        AmazonS3 s3Client=new AmazonS3Client(awsCreds);
        S3Object s3object = s3Client.getObject(new GetObjectRequest(versionConfig.getBucketKey(), versionConfig.getFileNameKey()));
        BufferedReader reader = new BufferedReader(new InputStreamReader(s3object.getObjectContent()));
        while (true) {
            String line = reader.readLine();
            //System.out.println(" "+line);
            if (line == null) 
                break;

            String[] fields=line.split("=");
            switch (fields[0]) {
                case "MAJOR":
                    versiondto.setMajor(fields[1] != null ? fields[1] : "0");
                    break;
                case "MINOR":
                    versiondto.setMinor(fields[1] != null ? fields[1] : "0");
                    break;
                case "HOTFIXPATCH":
                    versiondto.setHotFixPatch(fields[1] != null ? fields[1] : "0");
                    break;
                default:
                    LOGGER.info("INVALID/EXTRA FIELD IN VERSION.TXT FILE", fields[0]);
            }
        }
        s3object.close();
    } catch (AmazonServiceException ase) {
        LOGGER.error("Caught an AmazonServiceException from GET requests", ase);
    } catch (AmazonClientException ace) {
        LOGGER.error("Caught an AmazonClientException", ace);
    } catch (IOException ioe) {
        LOGGER.error("IOE Exception reading file from S3", ioe);
    }
    return versiondto;
}

I am trying to mock the AWS classes and run jUnit test against this method but am getting few exceptions. I have tried with mockito.spy, mocking aws classes. but still finding exceptions while testing the read-file method Following is the JUnit test:

public class VersionServiceTest {

@Mock
S3ObjectInputStream s3ObjectInputStream;

@InjectMocks
private VersionServiceImpl roleService;

@Mock
private VersionConfig versionConfig;

@Mock
private  S3Object s3Object ;

@Mock
BasicAWSCredentials awsCreds;

@Mock
private AmazonS3 s3Client;

@Before
public void init(){
    MockitoAnnotations.initMocks(this);
    when(versionConfig.getAccessKeyId()).thenReturn("AKIAJ6HT2QIXO452VW4A");
    when(versionConfig.getBucketKey()).thenReturn("version-test-ops/version");
    when(versionConfig.getFileNameKey()).thenReturn("version.txt");
    when(versionConfig.getSecretAccessKey()).thenReturn("6f+qJ/i0tQprEftE+pwa0CmnjTtdzoOYnuG0DGbN");
    awsCreds= new BasicAWSCredentials(versionConfig.getAccessKeyId(), versionConfig.getSecretAccessKey());
}

@Test
public void TestGetVersion() throws IOException {

    String bucket = "version-test-ops/version";
    String keyName = "version.txt";
    s3Object.setBucketName(bucket);
    s3Object.setKey(keyName);
    when(any(BasicAWSCredentials.class)).thenReturn(awsCreds);
    when(any(AmazonS3Client.class)).thenReturn((AmazonS3Client) s3Client);
    when(s3Client.getObject(any(GetObjectRequest.class))).thenReturn(s3Object);
    when(s3Object.getObjectContent()).thenReturn(s3ObjectInputStream);

    BufferedReader reader = Mockito.mock(BufferedReader.class);
    Mockito.when(reader.readLine()).thenReturn("ENV=DEV3", "MAJOR=0", "MINOR=0", "CRSPNG_VERSION_HOTFIXPATCH=384",
            "DATEOFDEPLOY=15-12-2017", "SPRINT=30");
    VersionDto dto= roleService.getVersionDetails();


}

}

fOLLOWING IS THE EXCEPTION:

java.lang.LinkageError: loader constraint violation: when resolving overridden method "com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(ILjava/net/Socket;Lorg/apache/http/HttpHost;Ljava/net/InetSocketAddress;Ljava/net/InetSocketAddress;Lorg/apache/http/protocol/HttpContext;)Ljava/net/Socket;" the class loader (instance of org/powermock/core/classloader/MockClassLoader) of the current class, com/amazonaws/http/conn/ssl/SdkTLSSocketFactory, and its superclass loader (instance of sun/misc/Launcher$AppClassLoader), have different Class objects for the type org/apache/http/protocol/HttpContext used in the signature
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:87)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:50)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:38)
at com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:260)
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:160)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:519)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:499)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:481)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:453)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:435)
at com.VersionServiceImpl.getVersionDetails(VersionServiceImpl.java:38)

Answer 1

The easiest way to mock S3 is to not use the S3 library directly. I use the FileSystem api introduced in Java 7 which was specifically designed to allow you to write your code once and use it with any file system. For unit tests I used the default FileSystem FileSystem.getDefault() which is just your local FileSystem. For production I create an instance of the S3FileSystem class which talks to S3. Upplication provides a very good implementation of the file system api for S3 here https://github.com/Upplication/Amazon-S3-FileSystem-NIO2

The advantage to this approach is that you don't have to mock anything to test your code, and it makes things very easy to test and debug.

Answer 2

I'm not sure why you're getting the error you're seeing but it looks like you're getting complication from not injecting mocks of the classes you need but trying to instantiate them directly. I would recommend you inject your AmazonS3 as follows:

class VersionServiceImpl implements VersionService {
    private AmazonS3 s3Client;

    public VersionServiceImpl(AmazonS3 s3Client) {
        this.s3Client = s3Client;
    }

    public VersionDto getVersion() {
        VersionDto versiondto = new VersionDto();
        try {
            S3Object s3object = s3Client
                    .getObject(new GetObjectRequest(versionConfig.getBucketKey(), versionConfig.getFileNameKey()));
            BufferedReader reader = new BufferedReader(new InputStreamReader(s3object.getObjectContent()));
            while (true) {
                String line = reader.readLine();
                // System.out.println(" "+line);
                if (line == null)
                    break;

                String[] fields = line.split("=");
                switch (fields[0]) {
                case "MAJOR":
                    versiondto.setMajor(fields[1] != null ? fields[1] : "0");
                    break;
                case "MINOR":
                    versiondto.setMinor(fields[1] != null ? fields[1] : "0");
                    break;
                case "HOTFIXPATCH":
                    versiondto.setHotFixPatch(fields[1] != null ? fields[1] : "0");
                    break;
                default:
                    LOGGER.info("INVALID/EXTRA FIELD IN VERSION.TXT FILE", fields[0]);
                }
            }
            s3object.close();
        } catch (AmazonServiceException ase) {
            LOGGER.error("Caught an AmazonServiceException from GET requests", ase);
        } catch (AmazonClientException ace) {
            LOGGER.error("Caught an AmazonClientException", ace);
        } catch (IOException ioe) {
            LOGGER.error("IOE Exception reading file from S3", ioe);
        }
        return versiondto;
    }
}

From there, your test only needs to care about passing an AmazonS3 mock. Also, the mocking declaration for the BufferedReader in the test would only work if you inject that reader in there. However, mocking the InputStream could be your best option. For that streaming mock you could go that way in your test:

    String expectedContents = String.join("\n", new string[] {
        "ENV=DEV3", "MAJOR=0", "MINOR=0", 
        "CRSPNG_VERSION_HOTFIXPATCH=384", "DATEOFDEPLOY=15-12-2017", "SPRINT=30"
        });
    InputStream testInputStream = new StringInputStream(expectedContents);
    when(s3ObjectInputStream.read(any(byte[].class))).thenAnswer(invocation -> {
        return testInputStream.read(invocation.getArgument(0));
    });

I have a more extensive example here You can find more help over there