This slide show is designed to run in full screen at 1024 * 768, the bottom part of some slides may be hidden at other resolutions. Also ensure the location bar, status bar and any other parts of the browser UI are hidden

About 3Q Software Solutions

3Q Solutions

Wealth Management and Financial Planning Solution Providers

3Q financial and wealth planning modules allow financial services companies to drive and enhance their sales process, delivering increased share of wallet and better client focused service.

3Q provides:

• Innovative financial product propositions
• Full financial planning capability
• Robust wealth management infrastructure

Founded by a group of financial services industry professionals, and skilled technologists, 3Q Solutions' builds visionary wealth management products that generate new ways to maintain and increase financial service revenues from their high net worth and mass affluent customer bases.

The ultimate vision of 3Q Solutions is to support client centric, sales driven business processes through clear, usable wealth management and financial planning solutions. This journey to full wealth management for many of our financial services clients is a step-by-step process, built first on 3Q’s compelling, client focused propositions. These propositions form into five foundation modules asset protection, asset accumulation, income creation, borrowing and cash management, which are at the base of any wealth management proposition.

Built with high net worth and mass affluent end customers in mind, 3Q’s model focuses on financial independence as a key indicator throughout it's products. The component-based products allow financial services companies to address immediate business goals first, and over time build their wealth management capability. 3Q’s modular, step-by-step approach means financial services companies can get immediate returns for each investment into their wealth management vision.

In Summary Anything that took a long time tended to get put off. If your build takes a long time to run, it wont be run often and problems wont be spotted as quickly. If your Acceptance tests take 20 minutes to run, they wont be run before a check in as soon as any time pressures mount. If your build process is not automated then natural human error will creep in to your finished builds. If Integration is cumbersome, developers will integrate as little as possible. If you are busy, you are unlikely to take the time to keep your customer in the loop. We have tackled all of these problems successfully using automation techniques.

This is what we did to improve the process, we will tell you how, it's not expensive, and you don't need to stop your development loop to implement it.

Automation like optimisation should not be premature, you need to ensure the investment you make will be paid back handsomely. Automation is a process.

Waiting for the build to compile, waiting for tests to run, waiting for acceptance tests to run. Waiting, Waiting, Waiting. This is time lost to you, your team and your company. Time everyone would rather you spent doing something else. But it's a nasty job and someone has to do it. The thing is that someone doesn't have to be human.

When it comes to software development a little time taken to script these tasks can free up a lot of time later on and provide added benefits besides.

We are going to walk you through a lot of small tweaks we have made to our development process through automation. None of these steps required a big investment in time, most where accomplished in under a day. And every one of them has paid back the investment many times over.

The trick is to recognize when your brain starts to hurt. When it comes to simple repeatable tasks, it is easy to just repeat them every time. Compiling, starting and stopping a server, waiting for tests to run, copying and pasting text from one file to another, all takes up a small amount of brain space. All together, these can make the job very laborious without individually seeming to be. When you're writing the boilerplate code for the millionth time, you've probably though it was a waste of time, but couldn't think of an immediate answer.

We are coming from a Java developers perspective, a lot of the tools mentioned are Java specific, equivalents are available for most popular languages and in most cases the tools are not that complicated and a home brew solution is a definite option.

We are not going to be covering the basics of continuous integration. We are assuming you all have working ant build scripts, a source control system, and a continuous integration tool like cruisecontrol, anthill, luntbuild or something similar. If you don't I can recommend reading the pragmatic starter kit, which will bring you fully up to speed. If you have no idea what continuous integration is, don't panic, I'm going to start off by walking you through our existing setup.

Our company is still somewhat in the startup phase, and we are constantly hunting for new customers, and revising our product line in view of the feedback we receive. This means we need to constantly supply marketing and sales with up to date builds for them to demonstrate to prospects. The feedback loop here can be very tight, suggestions from a meeting on a Monday might need to be demonstrable by Wednesday where practical. This means taking 2 days just to produce a build is unacceptable. Even producing a build every 3 or 4 hours takes away valuable development and testing time. We needed to reduce our build time dramatically

While we had an automated build it did not deliver a fully installable product. Portions of the build process where still manual. Whenever you introduce humans to the equation you introduce unreliability. A file might be missing from one build; a setting had not been updated in another. We needed to increase the confidence both we and our customers had in the build.

As builds where only really being delivered at the end of each iteration (approximately once every 2 weeks), our customer team could only give us feedback about once every 2 weeks. Two weeks is a long time when you are delivering software. We needed our customers to be able to tell us the minute we drifted off track. And we needed to let our customers know how development was going. Are we going to hit that ship date or not.

Our Estimation techniques, while good, where still far from perfect. As long as we spent one iteration after another on a single product, our estimation became progressively better. As soon as we switched back to another product in the line after a prolonged absence, we had forgotten how long the tasks involved really took. We needed a better record of our performance.

Continuous Integration tools include...

• CruiseControl
• AntHill
• LuntBuild

The best overall introduction to running a good automated build system is probably contained in The Pragmatic Starter Kit

• 1. Developer Integrates
• 2. Code is checked into CVS
• 3. Build server polls CVS for changes and starts a build.
• 4 - 6. Build server compiles, builds and tests the application. A burn test is run.
• 7. Build script assembles the application and creates a ready to run installer (build monitor constantly updates during all of this).
• 8. The finished installer is uploaded to an intranet
• 9. The customer team install and review the finished release and provide feedback to the developer team
• 10. Build can handle other administrative tasks, for example we compress and encrypt our source and backup to a remote server.
• The Entire process, from checking in to a new build being available on the intranet takes about 30 minutes. As there are 7 products in this cycle our average product is building in 5 minutes, including burn tests and automated acceptance tests. As we improve our build scripts this time is actually dropping rather than increasing with project size.

Automating delivery, meaning having the automated build produce a fully functioning, burn tested installer solved out problems with delivery time and confidence. The finished installer is automatically uploaded onto the company intranet, allowing our customers to try out the application several times a day and constantly provide us with feedback. We still need some manual testing for builds that reach the hands of end users, Chiefly platform testing. But we are working on automating a lot of this as well. The next big step for us is to automatically install the application on the customer teams machines as part of the build process.

The customer teams relationship with the developers became far more proactive. Rather than approaching us for status updates they where now providing corrections and updated stories. The development team now spends a minimal amount of time deliberately updating the customer team. As the build rarely produces a broken or non-functional product. The confidence level both the customer team and we have in the build has increased enormously.

While the automation of delivery gave our customer team a lot more information, a lot more could still be done. The customer team knew the status of the products at all times. But where still missing a lot of the big picture. How are the development team performing, are there any problems, how far through the current iteration are we? Yet again we automated passing this information to the customer team

The particular tools we used to accomplish this are referred to as Information Radiators. Information radiators are environmental sources of information. The information is pushed rather than pulled. The most common information radiator, which almost every XP team will be familiar with, is the card wall. Everyone who walks in the rooms can see how many cards are on the wall and get a sense of progress over time. Other tools we use include build monitors, systray applets, our intranet and several other toys.

The goal of using these tools is yet again to allow your customer team to become more proactive and reduce the communication burden on your team

This is our build monitor as we like to see it. A nice green screen. The monitor also displays the team's current velocity and the number of passing and failing acceptance tests in real time.

While the build is running we can see a yellow screen. The Current target area in the centre of the build monitor tells us what stage the build is currently in

Our dreaded nemesis, the angry monkey. This red screen tells us the build has failed. Complete logs of the build are on recorded on the server allowing us to quickly diagnose and remedy the problem that unleashed the angry monkey

As a backup to the build monitor we also have our system tray applet. As half the development team at any time have their backs to the screens, this allows them to view the current status by glancing at the bottom right of their screens, and optionally kick off a build at any time

Here you see the build monitor as it is situated in our offices. The important detail here is that it is right beside the coffee machine, the fridge, and the exit. This guarantees that everyone in the company walks by it several times a day. Our customer team begins to notice if the build has been broken for a long time, or velocity has been static for a while by osmosis. They don't have to actively seek out the information, they are bombarded with it. While they might not consciously look at the data, they become accustomed to it's natural rhythms and are quick to notice when the screen fails to match them

You may have noticed the second monitor beside our build monitor. The second monitor is our burndown, it switched back and forth throughout the day between two states. The first is a simple percentage measurement of how far through the iteration we are and is a direct reflection of velocity. This screen is a perfect example of what a low investment most of these tools are. From the idea to the working implementation took 28 minutes. Its actually a php web page with a little javascript to refresh it every 10 seconds.

The second mode it switches two shows a more detailed view of progress, if the green area is above the blue we are behind schedule, if it's below we are ahead. The blue area is the ideal iteration

In order to radiate all of this information, it has to be gathered. In the case of the build status, this is pretty simple. We simply collect the data from the build in webserver included with cruise control. As far as our velocity goes we use the simple tool above. In the past one of the developers sat down 2 or 3 times a week and put all of the data into an excel spreadsheet. This was a poor solution. It was a boring thankless task, and for the most part no one ever looked at the spreadsheet. The solution was this little tool. Every time a developer finished a card he takes a few seconds to log it's details. The app launches from the system tray and dumps the data into a database. It took only an hour to write the python script for this application. The end result is a source of raw data for tools like the burndown to use, and a very low cost when it comes to collecting the data.

For management and mining of the data we created a simple php site on the intranet. The burndown charts are simply part of this site. It also provides us with an ability to easily view the results of past iterations, allowing more accurate estimation.

The possibilities when it comes to information radiators are endless. This team used lava lamps; the advantage is that it takes a while for the green lamp to cool down and the red lamp to get started. They attempt to fix any broken build before the red lamp gets going.

Many development teams include a small message with each check in to source control describing the changes they have made. This team converted this information into an RSS feed which can be read by any newsreader, this gives the whole teams regular updates throughout the day as to what changes are being made and why. This example and many others are to be found at http://www.pragmaticautomation.com,The companion web site to the Pragmatic Automation book from the Pragmatic Starter Kit.

Taking the same idea a little further, this team has their automated build post it's results to a blog. A team diary. By allowing the developers to post messages to the blog as well, be it to record changes to the build process or carry out discussions. You have a permanent record of every change ever made to the build, and a record of every run in a very human readable form.

This radiator sums up the key to success with information radiators, they have to be in your face. An intranet or a spreadsheet on a network share will not succeed. People may check it at first, but over time interest will drop off. If the information is unavoidable in their environment, even if they do not pause to examine it in detail, they will notice abnormal changes it it's usual patterns. The second key is to make gathering the information as painless as possible if not completely automated. Otherwise as soon as things get busy this kind of administrative work will be the first thing to be put off.

Yet again the end result was a more proactive customer team. They could spot the development process going wrong as quickly as we could. And constant customer team oversight keeps us on our toes. We have our database of past iterations to help estimate future iterations, making it less likely that we will give cause for concern to the customer team in the first place. Everyone has access to this data, and it takes less effort than when only a handful of people had access. Repetitive donkey work has been eliminated and every wins are a result

If there's a portion of your domain that you find difficult of tedious to work with, you may be able to solve the problem by creating a small custom language that simplifies development in this area.

This depends on certain criteria:

The domain needs to be well-known and repeatable, and the domain needs to be mature enough that it makes the transition easily.

The foundation you build upon needs to be stable - you need to have already implemented the solution (boilerplate code) a hundred times

as a consequence, this is not a good solution to novel problems

This attack works best on problems which you have solved several hundred times, but due to complexity or tedium, the process of solving the problem still takes a significant amount of time ??? delete?

There are 2 main strategies available for tackling boilerplate and repetitive code. 1. Generating code , be it by simple GUI macros and scripts or code generation systems like cog or xdoclet. This is a technique we regularly use to overcome the lack of generics in Java, if you need the same functionality for several different types and a common interface is not a practical solution , the using code generation can allow you to provide a handful of variables to a script rather than writing 10's or even 100's of lines of code with only minor differences. 2. Create an mini-language/engine to achieve the same effect and store the multiple variations in data files. This approach is common in GUI development in the form of a markup language and process. A key feature that helped make out implementation a success is that it is easy to bypass the markup library and mix and match it with non markup based code , so in the event it was problematic or too time consuming to implement something for the markup engine the developers could drop back to raw SWT. **Small demo required here, with and without XMLGUI builder**

The bulk of developers I know only really know and use one language, be it Java, C#, C++ or whatever. I believe this is a bad thing. Every developer should have more than one tool in his toolbox. There is a family of languages that I believe are particularly suitable as a secondary language. These languages, commonly referred to as scripting or dynamic languages, are in many cases designed to integrate well with your main language. In particular, when you are not writing core or mission critical code, if you are writing build tools and scripts to massage data, your primary tool is probably not the best fit. Developers automate other people's processes for a living, but it's the cobbler's children who often do without shoes. We rarely examine and automate our own processes. In many cases the cost can genuinely be too high, we don't all write our own ides. But scripting languages usually have rich text and file handling capabilities and are particularly suited to reducing the repetitive and boring tasks developers are often faced with. This can reduce the cost to acceptable levels. Its time to give the cobbler's kids some shoes.

Programming Languages can be viewed as a tool for helping humans to think about solving problems. Some languages are better suited to some problems than others, regular expressions are great for matching complex string patterns but not much else. Ant is great for build scripts, but not for matching complex string patterns. Small mini languages can make solving difficult problems in specific domains relatively easy. Chapter 8 of 'The Art Of Unix Programming' contains a good discussion of a range of popular mini languages'

By making data processing scripts cheap, scripting languages make it possible to cheaply implement a lot of the tools we have looked at. The burndown chart, the card logger, the card database on the intranet, none took over 2 hours to implement with scripting languages. Environment Configuration alone could take that long with J2EE

NB By unsuitable for use, we are mainly referring to tool availability, refactoring browsers, cruise control and ant like build tools are not yet commonly available in most of these environments. While the performance of scripting languages is adequate for most tasks. In many domains it falls short.

Here are the homepages of some of the more popular scripting languages

• Python
• Perl
• Ruby
• PHP
• VBScript
• Groovy

import xml.dom.minidom
import sys

def find_control_names(node):
	
	if node.attributes and 'controlName' in node.attributes.keys():
		print node.attributes["controlName"].value
	
	for child in node.childNodes:
		find_control_names(child)

find_control_names(xml.dom.minidom.parse(sys.argv[1]).firstChild)

package com.threeq.spike;

import org.jdom.Element;
import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Iterator;

public class ControlNameExtractor
{
	public static void findControlNames(Element element)
	{
		if (element.getAttribute("controlName") != null)
			System.out.println(element.getAttribute("controlName").getValue());

		for (Iterator iterator = element.getChildren().iterator(); iterator.hasNext();)
			findControlNames((Element) iterator.next());
	}

	public static void main(String[] args) throws IOException, JDOMException
	{
		findControlNames(new SAXBuilder(false).build(new FileInputStream(args[0])).getRootElement());
	}
}

The first thing you might notice is that I had to drop the font size by a third to get the Java to fit on the page :)

For anyone who is curious I used GNU Source-highlight to pretty print the code. It happily handles

• Java
• Javascript
• C/C++
• Prolog
• Perl
• Php3
• Python
• Flex
• ChangeLog
• Ruby
• Lua
• Caml
• Sml
• Log

Which makes it particularly useful when comparing code in different languages.

Give the cobbler's kids some shoes (The poor little mites). If you have a language like this at your disposal tasks that you never considered automating will suddenly seem like obvious candidates. A few dozen simple scripts like the previously can collectively make a big difference to your productivity. That simple little script has halved the time it takes me to write interaction ATS.

This presentation was created using S5