3

I have a project where I need to be able to measure the efficiency in CPU and in memory space of functions I receive as text and compile at run-time using the Java Compiler API.

I am using ThreadMXBean to measure running time. Is there a better/more exact way? 

ManagementFactory.getThreadMXBean().getCurrentThreadCpuTime();

What I need is advice on how to measure the memory used. Ideally I would like to know about every assignment, and its size. But I would be alright with a precise count of the bytes used by the code.

Every piece of code need to be compiled, tested, run and evaluated automatically. So using a Profiler and look yourself is not an option. Any suggestion, possible method, resource or tools you might point me to is welcome.

I need to make that point clear. This is not a one off process. It is the main service of a web app that receives, compiles, analyzes and scores implemented functions on the fly. At pick I would expect a few 100 concurrent jobs going on. The process of monitoring once or manually with external tools is not my goal.

I repeat: I want to do this as a service. It is not for testing my apps or profiling them.

I do not bite (or down-vote since we are on SO), so any serious suggestion will earn up-votes.

status update: This project is on hold since I do not have a solid reliable way of doing the memory use measurements. I am currently looking at other languages/frameworks out of the JVM world that could this too. Even so I am still actively seeking advice and studying for this project.

le-doude
  • 3,345
  • 2
  • 25
  • 55
  • Don't reinvent the wheel - just use [Chronon](http://chrononsystems.com/) or some other profiler (like [VisualVM](http://www.codefactorycr.com/java-visualvm-to-profile-a-remote-server.html))? – Elliott Frisch Jan 05 '14 at 02:19
  • I am sorry @ElliottFrisch but you are off the mark here. I need the app to measure itself and return an output. I am well aware of VisualVM and other solutions that allow me to monitor my own applications myself. See comment thread on the most voted answer please. – le-doude Jan 05 '14 at 13:12
  • And you can't automate the profiler because? – Elliott Frisch Jan 05 '14 at 18:26
  • Please do introduce me to how to do that. After all I am still looking for the solution to this question. – le-doude Jan 05 '14 at 22:22
  • Basically, automate the build, deploy, execute, test phase. Add a timer at the test phase. Record the results (perhaps in a RDBMS). You'll need to explain more about what you're testing for a more meaningful answer. You might also leverage [Ganglia](http://en.wikipedia.org/wiki/Ganglia_%28software%29) - It depends on what you want to measure. – Elliott Frisch Jan 06 '14 at 00:51
  • 1
    **The point is not testing!** It is to do this As a Service. Users are "coders" that want to measure the various performances of a piece of code. They "input" to the app and the app compiles, tests, runs and measure the performances... and then output the performances (memory use and cpu use being the main ones). – le-doude Jan 06 '14 at 01:02

2 Answers2

3

I think using ThreadMXBean is the best way to achieve this, you can get all the info you need.

We are using this snippet to mesure the max memory used during an execution:

    try {
        PrintWriter out = new PrintWriter(System.out);
        List<MemoryPoolMXBean> pools = ManagementFactory
                .getMemoryPoolMXBeans();
        for (MemoryPoolMXBean pool : pools) {
            MemoryUsage peak = pool.getPeakUsage();
            out.printf("Peak %s memory used: %,d%n", pool.getName(),
                    peak.getUsed());
            out.printf("Peak %s memory reserved: %,d%n", pool.getName(),
                    peak.getCommitted());
        }
        out.close();
    } catch (Throwable t) {
        System.err.println("Exception in agent: " + t);
    }

Here is a usefull ressource for ManagementFactory use case.

Benoit Wickramarachi
  • 6,096
  • 5
  • 36
  • 46
  • Is there no way to have this for the specific thread in which I ran the received code? Or do I have to synthesize all the Mem Pools list to know? – le-doude Aug 09 '13 at 15:33
  • ThreadMXBean is connected directly to the JVM so you can't have the memory used by a specific thread only. You could maybe capture the memory allocation before and after having executed the piece of code to compute the memory usage. – Benoit Wickramarachi Aug 09 '13 at 15:49
  • Any hints on how to do that? Or do I have to write my own code analyzer? (I receive the code as plain text). – le-doude Aug 09 '13 at 17:34
  • Hum.. I think the best would be to compile the code and run it in a JVM, then monitor resources. – Benoit Wickramarachi Aug 09 '13 at 19:21
  • That is exactly what I can't do. My server will receive thousands of pieces of codes and will have to compile them, analyze them and score them real time. So monitoring is out of the question. – le-doude Aug 12 '13 at 09:59
1

It is not clearly stated, but it is implied in your question that the all functions will be running within the same jvm.

Although launching a separate jvm per function adds overhead I would seriously consider this option - it would make measuring memory and cpu usage much easier as you would be able to use the out of the box mx beans that collect per jvm stats and report stats back to the controlling process.

You may find that you need the isolation separate jvms provide.

Can you guarantee that none of the supplied functions will result in an infinite loop? Can you guarantee that none of them will exhaust all available memory or some other resource? Separate jvms would provide an effective sandbox.

henry
  • 5,923
  • 29
  • 46
  • I was thinking to launch the pieces of code in separate threads and monitor them. If they take too long or too much memory then I would kill them. Since those threads would only run that piece of code I though I could have enough to measure the variables I intended. – le-doude Aug 12 '13 at 13:39
  • 1
    The problem with this approach is that you can't reliably kill a thread in Java. See http://docs.oracle.com/javase/7/docs/technotes/guides/concurrency/threadPrimitiveDeprecation.html – henry Aug 12 '13 at 13:45