Transfer ORM and memory usage

Some people - myself included - have observed runaway memory usage and apparent memory leaks with applications built with certain combinations of CFML frameworks that include Transfer ORM. We spent a lot of time tuning the JVM and looking at code and database usage in our Broadchoice Community Platform (CMS), we worked with Mike Brunt on load testing and tuning (highly recommended - if you have any performance problems, get Mike on your case!) as well as working with Mark Mandel directly on Transfer itself. All that work led to a much more stable system and we decided to just continue investigating as a background task.

One of the key problems for us was that we could never reproduce the memory issues outside of our production system. We set up all sorts of test environments and ran load tests, to no avail. We tried a number of tests that eliminated pieces of the architecture to try to identify exactly what was the root cause. We use JMS to synchronize the cache on multiple instances so we turned that off, piece by piece, and eventually eliminated that as a source of the leak. That was good news - it meant we could continue using JMS to keep multiple instances synchronized!

Recently we've been running a number of Java profiling and memory analysis tools to look at JVM heap usage (we = one of my team, not me, so if you ask for details I'll have to get my colleague to provide them!) and established once and for all that we had some large cyclic graphics of objects that did not seem to be GC'd and that new cyclic graphs were being added fairly quickly. And those chains of objects were deep inside Transfer ORM. We'd finally "proved" that the problem was somehow in our usage of Transfer ORM.

I went back to Mark and asked about specific bug fixes in the caching machinery. It turned out that between the 1.1 RC1 release (which we had been using since it appeared) and the final 1.1 "Stable" release, Mark had made a couple of small tweaks to the two parts of the caching engine. We upgraded from 1.1 RC1 to 1.1 Stable and we've been watching memory usage in production ever since. So far - and I don't really want to jinx it by commenting - the results look very, very promising. Memory looks pretty stable after several days of runtime.

The irony is that I've always been an advocate of using the "BER" (Bleeding Edge Release) of frameworks and in pretty much the one case that I stayed with a particular release, I fell victim to bugs that were subsequently fixed!

So, if you're using Transfer and you're still on the 1.1 Release Candidate, you really should upgrade to the 1.1 Stable release!