How to use parallelism to speed up URL GET requests?

Question

The problem I am doing requires me to send requests to a website and check whether a specific password is correct. It is somewhat similar to a CTF problem, but I use brute force to generate the correct password key, as the site gives feedback whether a specific key is on the right track. In order for a key to be considered "almost-valid," it must be a substring of the correct key.

I have implemented this naively, but the intended solution uses simple parallelism to speed up the process. How would I accomplish this in Java?

import java.net.*;
import java.io.*;
import java.nio.charset.StandardCharsets;

public class Main {
    static boolean check(StringBuilder s) throws IOException{
        String test = "https://example.com?pass=";
        String q = URLEncoder.encode(s.toString(), StandardCharsets.UTF_8);
        URL site = new URL(test+q);
        URLConnection yc = site.openConnection();
        BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
        String inputLine;
        while ((inputLine = in.readLine()) != null) {
            //System.out.println(inputLine);
            if (inputLine.contains("Incorrect password!")) {
                return false;
            }
            if (inputLine.contains("the correct password!")) {
                System.out.println(s);
                System.exit(0);
            }
        }
        return true;
    }
    static void gen(StringBuilder s) throws IOException {
        if (!check(s)) {
            return;
        }
        for (int i = 33; i<127; i++) {
            int len = s.length();
            gen(s.append((char) i));
            s.setLength(len);
        }
    }
    public static void main(String[] args) throws IOException, InterruptedException {
        gen(new StringBuilder("start")); 
    }
}

EDIT: I have attempted to implement RecursiveAction & ForkJoinPool, but the code seems just as slow as the naive implementation. Am I implementing the parallelism incorrectly?

import java.nio.charset.StandardCharsets;
import java.util.concurrent.*;
import java.util.*;
import java.io.*;
import java.net.*;

public class cracked4 {
    static class Validator extends RecursiveAction{
        public String password;
        public Validator(String p) {
            password = p;
        }
        @Override
        protected void compute(){
            try {
                if (!valid(password)) return;
                System.out.println(password);
                ArrayList<Validator> futures = new ArrayList<>();
                for (int i = 33; i<127; i++) {
                    futures.add(new Validator(password + (char) i));
                }
                for (Validator t: futures) {
                    ForkJoinTask.invokeAll(t);
                }
                } catch (IOException e) {
                e.printStackTrace();
            }
        }

        public boolean valid(String s) throws IOException {
            String test = "https://example.com?pass=" + URLEncoder.encode(s, StandardCharsets.UTF_8);
            URL site = new URL(test);
            URLConnection yc = site.openConnection();
            BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
            String inputLine;
            while ((inputLine = in.readLine()) != null) {
                if (inputLine.contains("Incorrect password!")) {
                    return false;
                }
                if (inputLine.contains("the correct password!")) {
                    System.out.println(s);
                    System.exit(0);
                }
            }
            return true;
        }
    }
    public static void main(String[] args) {
        ForkJoinPool forkJoinPool = new ForkJoinPool();
        forkJoinPool.invoke(new Validator("cararra"));
    }
}

Furthermore, is there a certain UID serial I need? I researched about it, but I could not find a specific answer.

I hope this will help you. you can achieve it with multithread concept, https://stackoverflow.com/questions/2016083/what-is-the-easiest-way-to-parallelize-a-task-in-java — Arun Prasat, May 12 '21 at 03:20
Simple parallelism won't speed up the process. There is only one network. — user207421, May 12 '21 at 04:02
A "CTF" problem is a cybersecurity problem where a person is usually trying to retrieve a password/flag from a site/server using various techniques. — Krish, May 12 '21 at 11:42

score 1 · Accepted Answer · answered May 12 '21 at 20:30

Alright, I researched more about parallelism, and I decided on using ForkJoin / RecursiveAction. Using the parallelism allowed me to reduce my code execution time from 200 seconds to roughly 43 seconds (on my computer).

import java.util.*;
import java.util.concurrent.*;
import java.net.http.*;
import java.net.*;
import java.nio.charset.StandardCharsets;
import java.io.*;

public class cracked4 {
    static class Validator extends RecursiveAction {
        public StringBuilder password;
        public Validator(StringBuilder p) {
            password = p;
        }
        public static boolean valid(StringBuilder s) throws IOException, InterruptedException {
            HttpClient client = HttpClient.newHttpClient();
            String test = "https://example.com?pass=" + URLEncoder.encode(s.toString(), StandardCharsets.UTF_8);
            HttpRequest request = HttpRequest.newBuilder().uri(URI.create(test)).GET().build();
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            String text = response.body();
            if (text.contains("Incorrect password!")) {
                return false;
            }
            if (text.contains("the correct password!")) {
                System.out.println(s);
                System.exit(0);
            }
            return true;
        }
        @Override
        protected void compute() {
            try {
                if (!valid(password)) return;
                ArrayList<Validator> c = new ArrayList<>();
                for (int i = 33; i < 127; i++) {
                    StringBuilder t = new StringBuilder(password).append((char) i);
                    c.add(new Validator(t));
                }
                ForkJoinTask.invokeAll(c);
                }
            catch (IOException | InterruptedException ignored) {
            }
        }
    }
    public static void main(String[] args) {
        ForkJoinPool forkJoinPool = new ForkJoinPool();
        forkJoinPool.invoke(new Validator(new StringBuilder("start")));
    }
}

When I wrote this code initially, I used .fork(), but this did not help at all. In fact, it made it perform just as slow as the sequential code. I collected all of the Validator objects and I used ForkJoinTask.invokeAll(). This small difference resulted in an almost 200% speedup.

Deepak · Answer 2 · 2021-05-12T14:52:55.093

Hi please use RecursiveTask,ForkJoinPool for parallelism. Below code is not final modify on your own way.

public  class Test {

public static void main(String[] args) {
    ForkJoinPool forkJoinPool=new ForkJoinPool(parallelism);
    forkJoinPool.invoke(new PasswordKeyValidatorTask(new StringBuilder("start")));
}

public static class PasswordKeyValidatorTask extends RecursiveTask<StringBuilder> {

    private static final long serialVersionUID = 3113310524830439302L;

    private StringBuilder password;

    public PasswordKeyValidatorTask(StringBuilder password) {
        this.password = password;
    }

    @Override
    protected StringBuilder compute() {
        try {
            if (!valid(password)) {
                List<PasswordKeyValidatorTask> subtasks = new ArrayList<PasswordKeyValidatorTask>();
                for (int i = 33; i < 127; i++) {
                    PasswordKeyValidatorTask task = new PasswordKeyValidatorTask(password.append((char) i));
                    subtasks.add(task);
                }
                subtasks.stream().forEach(t -> t.fork());
       return subtasks.stream().map(t -> t.join()).findFirst().orElse(null);
            }
        } catch (Exception e) {
        }

        return password;
    }

    boolean valid(StringBuilder s) throws IOException {
        String test = "https://samplewebsite.com?pass=";
         String q = URLEncoder.encode(s.toString(), StandardCharsets.UTF_8);
        URL site = new URL(test + s);
        URLConnection yc = site.openConnection();
        BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
        String inputLine;
        while ((inputLine = in.readLine()) != null) {
            // System.out.println(inputLine);
            if (inputLine.contains("Incorrect password!")) {
                return false;
            }
            if (inputLine.contains("the correct password!")) {
                System.out.println(s);
            }
        }
        return true;
    }

}

}

What is the purpose of the serialVersionUID line? Secondly, I tested and slightly modified your code, but it simply stops after the first char is added to it. It doesn't keep going, and I am not sure why. — Krish, May 12 '21 at 12:40
return subtasks.stream().map(t -> t.join()).findAny().orElse(null); added that line. — Deepak, May 12 '21 at 14:54
ForkJoinTasks are Serializable, So that it canbe used in remote execution. — Deepak, May 12 '21 at 15:00
Speed depends upon available core and network speed. ForkJoinPool forkJoinPool = new ForkJoinPool(Runtime.getRuntime().availableProcessors()); — Deepak, May 12 '21 at 15:19

How to use parallelism to speed up URL GET requests?

2 Answers2