In App Engine specifically there are a number of unusual security limitations around File storage. I have found the best place to store resources securely is by using the bundle itself. If you're using the default Maven setup as produced by the appengine maven skeleton project this is as simple as placing the file inside of the appropriate resources directory

Once the p12 is in the correct location, you'll need to load it using the class loader's GetResourceAsStream function. Then when building the GoogleCredentials, don't use the documented setServiceAccountPrivateKeyFromP12File() function, but instead use the setServiceAccountPrivateKey() function and pass in the PrivateKey that you just constructed.
Additionally, you will most likely not want to use any of this functionality with a live appengine instance since Appengine already provides you with a much easier to use AppIdentityCredentials function in that case so you will probably want to detect whether or not your app is in production mode and only use the ServiceAccount when testing using localhost.
Putting all of these functions together yields the following function which works for me:
public static HttpRequestInitializer getDefaultCredentials() throws IOException
{
List<String> scopes = Arrays.asList(new String[] {DEVSTORAGE_FULL_CONTROL});
if (SystemProperty.environment.value() == SystemProperty.Environment.Value.Production)
return new AppIdentityCredential(scopes);
else
{
GoogleCredential credential;
try {
String p12Password = "notasecret";
ClassLoader classLoader = ServiceUtils.class.getClassLoader();
KeyStore keystore = KeyStore.getInstance("PKCS12");
InputStream keyFileStream = classLoader.getResourceAsStream("key.p12");
if (keyFileStream == null){
throw new Exception("Key File Not Found.");
}
keystore.load(keyFileStream, p12Password.toCharArray());
PrivateKey key = (PrivateKey)keystore.getKey("privatekey", p12Password.toCharArray());
credential = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId("YOUR_SERVICE_ACCOUNT_EMAIL@developer.gserviceaccount.com")
.setServiceAccountPrivateKey(key)
.setServiceAccountScopes(scopes)
.build();
} catch (GeneralSecurityException e) {
e.printStackTrace();
return null;
} catch (Exception e) {
e.printStackTrace();
return null;
}
return credential;
}
}