May 1999 - Present. ITG Australia

ITG Australia have a large amount of data in Sybase databases. I developed a Python extension module in C to interface to Sybase. This greatly simplifies the task of building programs which extract data from and load data into Sybase databases.

The module works on both Linux and Solaris, and has been released under a Free Software license, and can be downloaded from here.

ITG Australia has decided to standardise on Python as a scripting language. To help a number of people move to Python (including some Perl diehards), I wrote some extension modules which made Python faster than Perl for a number of key company tasks.

almanac

Every month an "almanac" is ditributed to all clients. Vast amounts of data are processed to produce the figures in the almanac. The data processing was previously performed by a suite of Perl scripts.

I pin-pointed the processing bottleneck and wrote a small C extension module for Python which encapsulated that processing. The equivalent, unoptimised, data processing programs in Python now run significantly faster than the highly optimised Perl programs.

tradebuf

ITGA provides set of web pages to their clients which provides a "Ready Reckoner" service which processes and reports on large amounts of post trade data on demand. When the Perl CGI scripts were replaced with Python scripts, there was a significant slowdown experienced.

I developed a C extension module for Python which transparently replaced the inner loop of the data processing Python scripts. A five-fold increase in speed was obtained bring responses back under 10 seconds.

htmlutil

A lot of my work at ITGA involved dynamic HTML generation for CGI. I developed a Python module which built on top of the standard HTMLgen Python module to provide common set of utilities for all web developers in ITGA.

ITGA have a software server which distributes live ASX trade data to a number of key systems within the organisation. I enhanced the server to allow ITGA to interactively split real trades in the server into an arbitrary number of virtual trades. Once a trade had been split into virtual trades, all dependant systems within ITGA would see the new virtual trades instead of the original real trade.

The ability to create virtual trades within the organisation has allowed some expensive problems in the internal procedures to be eliminated.

A number of the CGI tasks I performed at ITGA required persistent configuration data be maintained. To simplify the development of data entry interfaces, I developed a set of Python classes which provided a highly configurable framework for building web based data entry systems.

A new interface could be developed by simply building one form. Automatic defaults were provided for all database interfacing and CGI report forms.

ITGA use a back office system called Shares which performs a large amount of functions including produce contract notes. When producing a large number of contract notes, the system is very slow. This presents a problem to ITGA as they must report contract notes to their clients very soon after the market closes for the day. In some cases, it was just not possible to meet the reporting deadlines.

I developed a set of Python classes which accurately reproduced the contract note reports from the back office system, but in a fraction of the time. This code is now used for a lot of the contract note reporting as it eliminates the time critical aspect of the daily data entry and reporting processes.

Every day, a cron job recreates all contract notes for the previous day of trading and compares that to the contract notes actually produced in the back office system. Less than one contract note in a thousand has an error, in all cases so far, the error has been a single cent due to differences in rounding.

ITGA has a class of clients for which the shares traded over the day must sometimes be fairly allocated over a large number of accounts. For one stock, there may have been a number of trades of varying volumes at different prices. At the end of the day, we have to split integral numbers of shares from each of those trades amongst a large number of accounts while maintaining the same (or as close as possible) average share price over all of the accounts.

I developed a set of Python classes that quickly determine a fair and very close to optimal solution to the problem. The problem should be able to handle up to 1,000 accounts without performance problems. I also developed a program which uses these classes on a daily basis.

One of the third party systems used in ITGA maintains a database of all current market orders and an audit trail for those orders. Overnight these databases are exported, then all non-current data is purged from the database.

For compliance reasons, ITGA must keep historic order and audit trail data. I developed a set of programs that merged the daily database dumps into a single database which stores all order and audit trail data for as long as we require it. The interface to the historic data is a set of web pages which allows non-technical people inside the organisation to quickly locate and view orders via their browser.

The booking staff at ITGA perform a great number of manual data entry and reporting procedures after the market closes for the day. For this project, I developed a set of web pages which automated the reporting of share allocation confirmations.

One set of web based data entry forms allow the booking staff to customise the reporting method (fax or email) and parameters for each client. The main set of pages automatically query our back office system for data which can be reported. The booking staff then interactively select the allocations which need to reported, visually confirm that the report shown on their browser is correct, then initiate the electronic report delivery.

This streamlined procedure replaced a highly labour intensive and error prone procedure which involved cut & paste between a number of applications, printing and manual faxing.

The back office system used at ITGA is written in PROGRESS. This presents a problem since all of the ITGA in house developments use either Sybase or Ingres. A number of tasks require that complex queries be made on the back office system data. The approach to this problem has been to export selected data from the back office PROGRESS database and then load it into Sybase tables.

To facilitate the database export and load, I developed a set of Python classes and programs which read the PROGRESS data dictionary, then automatically produce scripts which build a parallel Sybase tables, and incrementally load the exported data. The automatic generation is extremely useful when an upgraded back office system is installed which changes the structure of the tables.

I developed a Python program which extracted doc strings from Python modules and output a cross-indexed HTML page which documented the classes and functions in those modules. This program was used to build HTML documentation for all Python modules that I developed at ITGA.

Top of Résumé