0

I've made a program that can read an xml file in and print the values from it back out. The xml file it handles is very large with like 40,000 entries of different data, and so I made a generic TreeNode class to handle each of those entries. However, as my program handles each entry of data by making a TreeNode object, it starts to consume a lot of RAM space until it runs out of space before it reaches the end. I know you can recycle your data structure, but I've looked online to find out how you can go about doing so, and as of yet I've not found a way of doing so. I know you can call recycle() on objects but for some reason it doesn't seem to work on my TreeNode. I was wondering whether anyone knew how I could go about implementing recycle in my code to help me overcome this issue, or any other way in which I can overcome my problem.

My code is below (I know it's not the best and most efficient way of doing it but it works):-

public class XML {

    TreeNode root;

    public XML(){

        try{

            InputStream file = new FileInputStream("/Users/Kevlar/Dropbox/PhD/Java/Metabolites/testOutput.xml");
            XMLInputFactory inputFactory = XMLInputFactory.newInstance();

            XMLStreamReader reader = inputFactory.createXMLStreamReader(file);

            reader.nextTag();

                do{
                    if(reader.getEventType() == XMLStreamConstants.START_ELEMENT && reader.getLocalName() == "metabolite"){
                        printChildren(mainElements(reader).children());

                    }

                    else{
                        reader.next();
                    }
                } while(reader.hasNext());


        }   catch (XMLStreamException e){
            System.err.println("XMLStreamException : " + e.getMessage());

        }   catch (FactoryConfigurationError e){
            System.err.println("FactoryConfigurationError : " + e.getMessage());

        }   catch (FileNotFoundException e){
            System.err.println("FileNotFoundException : " + e.getMessage());

        }
    }

private TreeNode mainElements(XMLStreamReader reader) throws XMLStreamException {

        Element rootElement = new Element();
        rootElement.setName(reader.getLocalName());
        System.out.println(rootElement.getName());
        root = new TreeNode(rootElement);

        int level = 1;

        do{
            int event = reader.next();

            if(event == XMLStreamConstants.START_ELEMENT) {

                Element element = new Element();
                element.setName(reader.getLocalName());
                TreeNode node = new TreeNode(element);
                level++;

                if(level == 2){
                    root.add(subElements(reader, node));
                    level--;

                }

            } else if(event == XMLStreamConstants.END_ELEMENT){
                level--;
            }

        } while(level > 0);

        return root;
    }

private TreeNode subElements(XMLStreamReader reader, TreeNode node) throws XMLStreamException {

    int level = 1;

    do{
        int event = reader.next();

        if(event == XMLStreamConstants.START_ELEMENT) {

            Element subElement = new Element();
            subElement.setName(reader.getLocalName());
            TreeNode subNode = new TreeNode(subElement);
            level++;

            if(level == 2){
                node.add(subElements(reader, subNode));
                level--;
            }

        } else if(event == XMLStreamConstants.CHARACTERS && !reader.isWhiteSpace()){

            node.getElement().setValue(reader.getText());

        } else if(event == XMLStreamConstants.END_ELEMENT){
            level--;
        }

    } while(level > 0);

    return node;
}

private void printChildren(TreeNode[] children) {

    for(int i = 0; i < children.length; i++){
        TreeNode node = children[i];
        if(node.hasChildren()){
            System.out.println("Element:- " + node.getElementName());
            printChildren(node.children());
        }

        else if(!node.hasChildren()){
            System.out.println("Element:- " + node.getElementName());
            System.out.println("Value:- " + node.getElementValue());
        }
    }



}

public static void main(String[] args) throws XMLStreamException{

    XML test = new XML();

}

}
user2062207
  • 955
  • 4
  • 18
  • 34
  • 1
    The JVM will recycle automatically, no need to tell it to do so. Just drop all references to the object that is no longer required, and let the JVM manage the memory. – Chris K Nov 24 '14 at 10:32
  • Make sure not to have references to objects you don't need and JVM will remove them automatically. – Andrey Nov 24 '14 at 10:33
  • But don't you drop the reference to the object when you make a new object out of it? So for my treenode, once it's done being populated with one entry, the program then makes a new treenode, thereby dropping references to the the previous treenode object? Or have I got that completely wrong? – user2062207 Nov 24 '14 at 10:42
  • 1
    `node.add(subElements(reader, subNode));` does not look like it is dropping references to me. – Chris K Nov 24 '14 at 10:45
  • Are you using a SAX, DOM or other api for parsing? DOM by definition will load the entire xml document into memory. – Chris K Nov 24 '14 at 10:45
  • Perhaps your next step should be to fire up your application using [YourKit](http://www.yourkit.com/). It will let you walk the objects in memory, and inspect what the true relationships are. – Chris K Nov 24 '14 at 10:47
  • Do you, or do you not, want all the data loaded in memory at one time? – user253751 Nov 24 '14 at 10:49
  • I'm using StAX for parsing, it parses a section of the data into a TreeNode (i.e. each entry at a time). Also with the node.add part, would doing this:- TreeNode newNode = new TreeNode(); newNode = node; overcome that problem? As in initializing a new variable to that of the node within that method so a new node is generated every time that method is called? – user2062207 Nov 24 '14 at 10:49
  • So it would be newNode.add(subElements(reader, subNode)); instead basically – user2062207 Nov 24 '14 at 10:50
  • And I don't want to load all the data into the memory at one time, just sections of it. I'll give yourkit a go as well thanks! – user2062207 Nov 24 '14 at 10:52
  • Probably should of mentioned that my level of coding ability is very minimal, and yourkit is confusing as hell to use :( – user2062207 Nov 24 '14 at 11:21

0 Answers0