byte[] sending in UDP java unable to give whole value

Question

I have this small error about my output from Client server not working properly. The issue is that the output sent over from my server to client will not always be the same after decryption. In this case, the resulting decryption is the same, only up till a certain amount of length.

Below are the output, follow by the source code:

Server
Starting the socket server at port:9876
Waiting for clients...

GxModP = 2819481724922694722202990425920759863750647735434635820227489190230872167846391528548356160689879969739097933352014135386311977602583588682340466075235457

encrypted = Y:!T¤]®ð]áfyLkåª hÛÒ2{ø2/Z°ÿFär8 icÂÂñýÜå¼Níxv_Ö3?»>@×âÑÛm4ÁônûZùGÃêa¼/d
×Q¨_¬ù6øY¾½¡pOY÷?x¼ù>öMÀ|c´áúê±ôpZþÏÈy­9½4S


decrypted = 2819481724922694722202990425920759863750647735434635820227489190230872167846391528548356160689879969739097933352014135386311977602583588682340466075235457


Client
Attempting to connect to localhost:9876
Connection Established

User input : Y:!T¤]®ð]áfyLkåª hÛÒ2{ø2/Z°ÿFär8 icÂÂñýÜå¼Níxv_Ö3?»>@×âÑÛm4ÁônûZùGÃêa¼/d

Gx mod P : 281948172492269472220299042592075986375064773543463582022748919023087216784639152854835616068987

What I did here was to place encrypted into client input and decrypt the encrypted message in order to arrive at GxModP. However, it seems that the value of GxModP is only same till a certain length. Can anyone tell me what mistake have I made that causes this?

I have attached source codes below for reference

    private static String algorithm = "RC4";

            public static byte[] encryptRC4(String toEncrypt, String key) throws Exception {
              // create a binary key from the argument key (seed)
              SecureRandom sr = new SecureRandom(key.getBytes("ISO-8859-1"));
              KeyGenerator kg = KeyGenerator.getInstance(algorithm);
              kg.init(sr);
              SecretKey sk = kg.generateKey();

              // create an instance of cipher
              Cipher cipher = Cipher.getInstance(algorithm);

              // initialize the 

cipher with the key
          cipher.init(Cipher.ENCRYPT_MODE, sk);

          // enctypt!
          byte[] encrypted = cipher.doFinal(toEncrypt.getBytes("ISO-8859-1"));

          return encrypted;
        }

        public static String decryptRC4(byte[] toDecrypt, String key, int length) throws Exception {
          // create a binary key from the argument key (seed)
          SecureRandom sr = new SecureRandom(key.getBytes("ISO-8859-1"));
          KeyGenerator kg = KeyGenerator.getInstance(algorithm);
          kg.init(sr);
          SecretKey sk = kg.generateKey();

          // do the decryption with that key
          Cipher cipher = Cipher.getInstance(algorithm);
          cipher.init(Cipher.DECRYPT_MODE, sk);
          byte[] decrypted = cipher.doFinal(toDecrypt, 0, length);


          return new String(decrypted, "ISO-8859-1");
       }

Server code:

private void sendKey(Socket client) throws IOException {
    //calculate GxModP
    //encrypt + send over.

    String gXmodP = generateGxModP(g, p);
String temp = "";
System.out.println("GxModP = "+gXmodP);
try
{
    encryptedKey = EncryptionScheme.encryptRC4(gXmodP, password);
}catch(Exception e){}

String encrypted = new String(encryptedKey, "ISO-8859-1");

byte[] temp2 = encrypted.getBytes("ISO-8859-1");
System.out.println("encrypted = "+encrypted);
try
{
    temp = EncryptionScheme.decryptRC4(temp2, password, temp2.length);
}catch(Exception e){};
System.out.println("decrypted = " + temp);

BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(client.getOutputStream()));
writer.write(encrypted);
writer.flush();
writer.close();
}

Client code

public void readResponse() throws IOException
{
    String userInput;
    BufferedReader stdIn = new BufferedReader(new InputStreamReader(socketClient.getInputStream()));
    String gXmodP = "";
    /*
    System.out.println("Response from server:");
    while ((userInput = stdIn.readLine()) != null) {
        System.out.println(userInput);*/ 
    //instead of reading response i will decrypt and print out

    userInput = stdIn.readLine();
    System.out.println("User input : " + userInput);
    byte [] toDecrypt = userInput.getBytes("ISO-8859-1");
    try
    {
        gXmodP = EncryptionScheme.decryptRC4(toDecrypt, password, toDecrypt.length);
    }catch(Exception e){};
    System.out.println("Gx mod P : " + gXmodP);
}

Answer 1

The way you treat the binary result of the encryption output is error-prone. You should not directly convert the byte[] to a String and transfer it as characters (use an encoding like Base64 or stick completely to a binary transmission).

In your specific example the problem comes from the fact that the encryption result contains a line break character / \n in binary. So readLine(); on the client side cuts the binary string there and hence your decrypted text is cut, too. Otherwise your encryption code runs perfectly fine.

I won't comment on using RC-4, though.