Some Code Statistics for Broadchoice Workspace

Now that the Broadchoice Workspace has been officially released, I thought I'd post a quick entry with some code statistics. I ran the codebase through a line counting tool that gives a breakdown by file, folder, language, etc. We all know that lines of code is not a very good way to judge quality or effort, but it is still interesting in a general way. This is only lines of code, all blank lines and comments were excluded:

  • Total Lines of Code: 50,125
  • Total Files: 633
  • Lines of MXML: 10,784
  • Lines of ActionScript: 12,675
  • Lines of CFML: 7,730
  • Lines of Groovy: 8,878
  • Lines of XML: 1,757
  • Lines of CSS, HTML, JavaScript, etc.: 7,612
  • Lines of SQL: 1,089
  • Lines excluded as comments or blank: 15,778

Is it ColdFusion or Groovy?

Here at Broadchoice we've been working on our new product for close to three months now. I've gotten a crash course in Groovy, Spring, and Hibernate, all of which are slightly familiar to me from my time with ColdSpring, Transfer, and Java. I'm really digging these new technologies, but of course, I'm a die hard ColdFusion user at heart. A few weeks backs I needed to work on a quick iPhone web application prototype. Joe set up the connection and all I had to do was write CFML (lucky me). Little did I know that along with setting up basic CFML compatibility on the server, Joe also added one of the slickest things I've seen ever in my life as a developer.

The application I'm working on is pretty simple, but I wanted to use Model-Glue 3 for it since - well - you never know when a simple prototype will turn into a large, complex application. The following code snippet comes from a controller used to load bookmarks:


<cffunction name="getBookmarks" output="false">
<cfargument name="event" />
<cfset var u = arguments.event.getValue("currentuser")>

<!--- In theory currentuser should always exist, but... --->
<cfif not isSimpleValue(u)>
<!--- space only? --->
<cfif arguments.event.valueExists("space")>
<cfset arguments.event.setValue("bookmarks", beans.contentService.findContentBySpace(arguments.event.getValue("space"),["Bookmark"]))>
<cfelse>
<cfset arguments.event.setValue("bookmarks", beans.contentService.findUsersContent(u,["Bookmark"]))>
</cfif>
</cfif>

</cffunction>
Nothing in the above code should be too odd or even that exciting really. I'm using a service (injected via the beans scope in Model-Glue 3) to retrieve my content. My controller begins with...


<cfcomponent output="false" hint="I am a Model-Glue controller." extends="ModelGlue.gesture.controller.Controller"
beans="contentService,config"
>

Note the beans value specifies which beans configured in ColdSpring should be passed to my controller. I love this new automation in Model-Glue 3. If you open up my ColdSpring.xml file, you will see my beans configured:


<bean id="config" class="ModelGlue.Bean.CommonBeans.SimpleConfig">
<property name="config">
<map>
<entry key="profileimageroot"><value>/some url you dont need to know</value></entry>
<entry key="filesroot"><value>/something else you dont need to worry about</value></entry>
<entry key="perpage"><value>10</value></entry>
</map>
</property>
</bean>

Hmmm. Ok, there's my config bean, wheres my contentService bean? Oh yeah - here it is:


<bean id="contentService" class="com.broadchoice.bcp.services.ContentService">
<property name="dao" ref="contentDAO"/>
<property name="sessionFacade" ref="sessionFacade" />
<property name="sharedFileGateway" ref="sharedFileGateway" />
<property name="eventMessageDispatcher" ref="eventMessageDispatcherService" />
</bean>

You can find the above in our Spring file. Just in case the above didn't quite make get through, let me make it real nice and obvious:


<cfset arguments.event.setValue("bookmarks", beans.contentService.findContentBySpace(arguments.event.getValue("space"),["Bookmark"]))>

The above line of CFML is using a bean injected from Spring, not ColdSpring, and points to a Groovy file. This is the same Groovy code used to drive the AIR application. No createObject("java") here baby. I'm using the Groovy code just like it had come from a CFC. This is done using a ColdSpring/Spring adapter that Joe wrote. It lets ModelGlue get Java-based beans from Spring. (I'm simplifying it here a bit, but hopefully you get the significance of that!) Once that connection is done, you can easily get beans from the Spring file and inject them into your controller. So for example, when my UserController wanted do handle login, I added: beans="userService,config" and then simply ran a userService.validateAndLoad on the authentication information.

It's always been easy to use Java from CFML, but never has been so sexy! Now to be fair, I did have to use a JavaCast once or twice, but not nearly as much as I would have thought.

Behind The Curtain ... continued

A few days back Sean gave us all a sneak peak of the Broadchoice Workspace. He also mentioned we're all very busy lately, and guess what ... the Broadchoice Workspace is not the only application we're working on. Today I'm going to introduce the Broadchoice Workspace Analytics platform.

[More]

AIR File Download Gotcha

If you happen to work with Adobe AIR at some point, and you want to download a file from a server, you may run into this helpful error:

I/O error 2038

Not very helpful. After a lot of digging and trying out solutions, I think I've figured it out. I was doing something like this in a Delegate object:


public function downloadFile( fileName:String ):void {
    var req:URLRequest = new URLRequest( this.sharedFileDownloadURL + fileName );
    var file = new File();
    file.addEventListener( Event.COMPLETE, saveCompleteHandler );
    file.addEventListener( IOErrorEvent.IO_ERROR, ioErrorHandler );
    file.download( req, fileName );
}

The problem seems to be the scope of the File object that is used to perform the download. Because it is locally scoped to that method, when the download completes and the completion event is announced, the file object is essentially gone. I'm not sure if AIR is garbage collecting it, or if the variable is just not accessible because it is function-scoped. Either way, I got this error over and over and was pulling my hair out.

The solution is to not var scope the new File object. Unfortunately that seems to open up potential concurrency issues. Because the Delegate object is essentially a Singlton, if I add the file as a public property of the Delegate, and more than one upload goes at the same time on the client, it will get overwritten. That's not good.

I got around it by creating an Object to wrap around my file called SharedFile. This object has a public property which is a File object. In that case, I can do something like this:


public function downloadFile( sharedFile:SharedFile ):void {
    var req:URLRequest = new URLRequest( this.sharedFileDownloadURL + sharedFile.name );
    sharedFile.file = new File();
    sharedFile.file.addEventListener( Event.COMPLETE, saveCompleteHandler );
    sharedFile.file.addEventListener( IOErrorEvent.IO_ERROR, ioErrorHandler );
    sharedFile.file.download( req, sharedFile.name );
}

So I'm setting the sharedFile.file property to a new File, then using that file object to perform the upload. The I/O error vanishes because the file object is not locally-scoped, and so a reference to it still exists when the completion event fires. Again, I'm not sure of the exact underlying cause of the problem, whether a function-scoped file object is simply not visible when the completion event fires, or whether the AIR runtime is garbage collecting the variable before the completion event fires. Either way, this seems to fix it!

Better Living Through Transfer and ColdSpring

We have a live system with customer data and a new requirement comes along that a particular piece of customer data must be encrypted in the database from now on. We already encrypt some columns (using Triple DES - which you might have guessed given the recent posts on my blog and here about mimicking ColdFusion's encryption in Java/Groovy). What is the smallest possible code change to ensure that as any user updates their data in future, this item will automatically be encrypted - whilst still handling the case of legacy data being unencrypted?

We use Transfer for all our persistent business objects and almost all of our business objects have a decorator defined (for validation or some additional business logic). We also use Brian's TDOBeanInjectorObserver to automatically inject services into our business objects - just add a setter for a service and the bean injector takes care of the rest.

Here's the bean injector definition in our ColdSpring file:


<bean id="transferObjectInjector" class="coldspring.transfer.TDOBeanInjectorObserver">
    <constructor-arg name="transfer"><ref bean="transfer" /></constructor-arg>
    <constructor-arg name="suffixList"><value>service,datasource</value></constructor-arg>
    <constructor-arg name="debugMode"><value>true</value></constructor-arg>
</bean>
Normally you would declare it non-lazy but we already do other non-lazy initialization so in our ColdSpring factory initialization code, we do this:

<cfset bf.getBean("transferObjectInjector") />
to force the bean injector to be initialized which, in turn, registers itself as a Transfer event listener (so that it can intercept object creation). The suffixList specifies that any set*Service() method or set*Datasource() method on the business objects managed by Transfer should be matched to beans defined in ColdSpring and injected.

So how do we add the on-demand encryption to our business object's data?

[More]

AIR and Groovy via Spring

As a fun way to while away a Saturday evening, I decided to look at Joe's Behavioral Analytics backend code to see if I could create a Flex-based AIR application that talked to services written in Groovy. There's not really much documentation out there to help so it was a very hit and miss experience but eventually I had a simple AIR application making a RemoteObject call to a Groovy class and getting data back. I'm going to have to talk to Joe about cleaning up the installation I've ended up with but here are the highlights (and I'll blog more about this once I have all the pieces cleaned up):

  • JBoss 4.2.3 GA installed as a Server in Eclipse (using the Web Tool Platform JEE stuff)
  • Spring framework 2.5.5
  • A Spring-based Flex factory that allows Flex Remoting to talk to objects via Spring (author Jeff Vroom)
  • Groovy - set up per Joe's recent instructions
  • BlazeDS providing the Flex Remoting service
Create a Dynamic Web Project in Eclipse and then unzip BlazeDS into it (per the BlazeDS installation docs). Add Spring and the flex.samples.factories.SpringFactory Java class. Configure Flex Remoting according to Lin's instructions to add the Spring factory:

    <factories>
        <factory id="spring" class="flex.samples.factories.SpringFactory"/>
    </factories>
and the context loader:

    <!-- Spring -->
    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/applicationContext.xml</param-value>
    </context-param>
    
    <!-- load Spring's WebApplicationContext -->
    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>
and the destination for the RemoteObject:

<destination id="place">
        <properties>
            <factory>spring</factory>
            <source>placeholder</source>
        </properties>
</destination>

Create a Flex AIR project and define a RemoteObject that refers to the destination you just defined:


    <mx:RemoteObject id="bcp"
            destination="place"
            endpoint="http://127.0.0.1:8080/bcpbackend/messagebroker/amf"
            >

        <mx:method name="test" result="testResult(event)" fault="testFault(event)"/>
    </mx:RemoteObject>

Add the placeholder bean definition to your applicationContext.xml file:


    <bean id="placeholder" class="com.broadchoice.bcp.Placeholder"/>
com.broadchoice.bcp.Placeholder is a Groovy class that contains a test method (for testing it just takes a string and returns that string plus some additional text).

Living in a Cluster

The Broadchoice Collaboration Platform (BCP) is deployed on a cluster of servers and this has a number of interesting implications for the design of the application as well as the actual deployment process and file system structure used.

I'll be posting several entries on clustering considerations but I wanted to start with something that surfaced with the recent launch of this blog. BlogCFC is part of our standard SVN repository (I'll also be blogging about our source code control processes) and so it is also deployed to the same cluster as the BCP. Each server runs the same code from its own local file system. BlogCFC allows authors to upload images and files that are used as part of the blog entries. If you simply deploy BlogCFC onto each server in the cluster and then create a blog entry and upload an image, that image will be stored directly on the file system of the server on which your request is processed (in fact, the server on which your entire session is processed, since we use "sticky session"). The other servers in the cluster won't know about the image. For user requests that come in to that first server, the image will be served correctly. For user requests that come in to the other servers, the image will be missing.

The BCP has the same issue - it allows authors to upload CSS, images and documents - but we have to ensure that all these uploaded assets are available to all servers automatically and immediately. Our approach was to design the application in such a way that shared assets are stored in specific directory trees that contain nothing but shared assets. We have a NAS - Network Attached Storage - which is mounted to every server as /var/www/html. That contains a documents directory and a custom assets directory - into which all uploaded content is placed. Symbolic links are used to "map" those shared directories to the appropriate place in the deployment directory tree:

/var/www/production/lib -> /var/www/html/lib
/var/www/production/wwwportal/custom -> /var/www/html/custom
We deploy our source tree to /var/www/production (direct from SVN) with lib and custom ignored by SVN. Each server then shares the same uploaded content without needing to use different file paths to how we would deploy to a non-clustered server.

Tonight, I applied the same fix to BlogCFC, adding blog_images and blog_enclosures directories on the NAS and adding symbolic links back into the BlogCFC deployment tree (as wwwblog/images and wwwblog/enclosures respectively). We have not yet dealt with making blog.init.cfm cluster-safe or the XML file generated by the pod manager. For both of those, we actually keep the files under SVN and handle changes as part of a managed process (i.e., by tickets in Trac).

This is just one of many things that need to be considered when designing applications for clustered environments. I'll be blogging about other clustering issues over the next few weeks.

Numbers are a useless emotion: A year of traffic in one single impression

They say a picture is worth more than thousand words and yet in almost any web analytics package all we get are numbers. Ok, they have pie charts, bar charts and the obligatory map overlay, but if you have over a 100 reports you need to be able to fill up the screen, right? For the past year I've been using a lot of different web analytics packages and I've become more and more convinced that numbers tell you very little, especially if they can not be compared with other numbers.

I'm all about shiny stuff and pretty pictures so last night I tried to compile a full year of traffic in one single picture and this is the result:

[More]

Creating an Eclipse-based Groovy Development Environment for TDD

Within the system I'm building at Broadchoice, we're exploring the use of Groovy, Spring, and Hibernate in combination. The first step in building a system using this stack is creating an Eclipse-based environment in which we can write and test our code. To help people do this, I've published a Google document entitled Creating an Eclipse-based Groovy Development Environment for TDD.

As it doesn't contain any "technical secrets" (we don't really have the concept of a technical secret at Broadchoice!), I thought it'd be handy to share it with the world.

I've still got more to document, including how to integrate TestNG tests with Spring and how build (and effectively test!) a Hibernate-enabled Groovy+Spring project. A lot of this is the result of reading multiple books and a good deal of Googling / piecing together information, so I hope you enjoy it in distilled, ready-to-consume formats!

A bit of practical Groovy

While working on server-side components for the Broadchoice Behavior Analytics product, I've been using Groovy to write Java classes. I'm finding that it has many parallels with ColdFusion when it comes to being a productivity layer on top of Java.

A great example little command-line tool I wrote for personal purposes. I needed to query a Subversion server for all files modified or added in a given array of changeset numbers, building a .zip file containing the entire bundle. 120 lines of Groovy (including nice formatting and comments!) was all it took.

Here's a few excerpts:

Executing SVN at the command line for each revision number and parsing out the list of changed or added files from the resultant XML:


revisions.each{revisionNumber ->

    System.out.print("Getting log for revision: " + revisionNumber + "\n")
    
    cmd = "svn log -r " + revisionNumber + " -v --xml --username " + username + " --password " + password + " " + url;
    svnProcess = cmd.execute()
    svnProcess.waitFor()

    new XmlParser().parseText(svnProcess.in.text).logentry.each { entry ->
        revisionFiles[revisionNumber] = entry.paths.path.findAll {
            return (
                it.'@action'.contains('M')
                || it.'@action'.contains('A')
            )
        }.collect { path ->
            path.text()
        }
    }
}

Yep, that's all it takes to execute a command line process, parse its result into an XML document, and then parse that document into a local Map of file!

At the end, I needed to create a zip file of a given directory. Using the AntBuilder (which dynamically interprets method calls as Ant tasks), it was a no-brainer:

Using AntBuilder to build a ZIP file of all files within a given directory.


new AntBuilder().zip(
        destfile : ("build." + version + ".zip"),
        basedir : "./working/" + version
)

Groovy is...groovy. I've always loved ColdFusion for its practicality, and Groovy is providing many of the same features when I need to write raw Java or create command-line tools such as this utility!

BlogCFC was created by Raymond Camden. This blog is running version 5.9.1. Contact Blog Owner