Grails, Inserting lots of data using withTransaction results in OutOfMemoryError

Ted Naleid wrote a great blog entry about improving batch performance. Including here as a reference.

This is a common issue with all hibernate applications and it is caused by the growth of the hibernate session. I'm guessing that the grails console holds a hibernate session open for you in a similar way to the 'open session in view' pattern that I know it uses in for normal web requests.

The solution is to get hold of the current session and clear it after each batch. I'm not sure how you get hold of spring bean using the console, normally for controllers or services you just declare them as members. Then you can get the current session with sessionFactory.getCurrentSession(). In order to clear it just call session.clear(), or if you what to be selective use session.evict(Object) for each Person object.

for a controller/service:

class FooController {
    def sessionFactory

    def doStuff = {
        List batch = new ArrayList()
        for (each person in legacy phone book) {
            // Construct new Grails domain class from legacy phone book person
            Person person = new Person(...)
            batch.add(person)
            if (batch.size() > 500) {
                Person.withTransaction {
                    for (Person p: batch)
                        p.save()
                    batch.clear()
                }
                // clear session here.
                sessionFactory.getCurrentSession().clear();
            }
        }
        // Save any remaining
        for (Person p: batch)
            p.save()
        }
    }
}

Hope this helps.