Project Code Name: TSDB. Timeframe: 1998-2000.
Project Background
The Partner
World Systems Luxembourg Ltd. is a consultancy company with branches both in Luxembourg and London-UK. During 1997-2000 they were contracted by RNCS for the implementation of a PHARE project, particularly the development of a full data-warehouse system entitled TSDB – Time Series Data Base.
The Project
The full project was outsourced to 4E Software Ltd. during 1998-2000 timeframe. Our company was in charge of all projects aspects like:
• Project Management
• Software Project Development Life-Cycle (analysis, architecture, coding, testing and implementation)
• Project QA
• System maintenance
The Time-Series Database System was designed as a huge custom-designed data-warehouse to store in a centralized environment and organize the main statistical data and metadata in the RNCS – the final customer of World Systems Luxembourg Ltd.
The main purposes of the system were:
• to ease the informational flows between RNCS compartments, through direct access for study and analysis to an unique, reference collection of statistical data;
• to satisfy external users' demands with more efficiency, by having information from every statistical domain in one place;
• to minimize interpretation errors of statistical data, by integrating them with metadata;
• to allow global evaluation of the harmonization degree between the RNCS statistical system and the European Union standards.
As direct functions, by means of the system one may produce derived indicators from existing ones, make statistical analysis of data, economical and social studies and analysis, deliver statistical data to external users (such as various ministries, public institutions, press, etc).
The Success Story
The main features of the system are:
• Business Intelligence / data dissemination using OLAP technology;
• Business Intelligence / data dissemination using GIS technology - mapping data hypercubes on geographical information;
• the management of the database that should contain all statistical data;
The design of the system started in late 1997. Several options were taken into consideration, and several architectures were issued, each of them with its inherent problems. 4E Software Ltd. was contacted for the full project outsourcing 16 months after project start-up in late 1998. Our team decided to review the entire work that has been done in the TSDB system and stress-test each designed component of the system. Finally we came with a fully new architecture of the TSDB project. Although the principle remained that the system is to be designed as a client/server architecture, there were discussions of how this would be done, technologies involved, design details. Main issues were two-tier vs. three-tier architecture, location of temporary data, object streaming, object design.
Development details
The design has been entirely done using UML - design tools used were Oracle CASE, MicroGold WithClass, and Model Maker. As a result, a two-tier client/server architecture was implemented, based on Oracle 7.3 as the database server, implemented both on Windows NT (for testing and development purposes) and Unix (working database). The client was implemented using Borland Delphi 3.0 and 4.0 (timeframe issues). The client communicates with the server through Borland Database Engine and Oracle client - SQL *Net -, and the local data (temporary information) is stored/processed in temporary virtual tables at the system core level – using a custom developed database layer that allows the client application to use huge amounts of data needed for report generation without loading the network or other hardware resources. The ODBC-based connection had to be early abandoned because of the major limitations with Oracle.
The application logic had to be divided between the two tiers upon efficiency criteria. There had to be taken into consideration both the fact that the database is especially large and growing, and the big amount of data to be transferred between the client and the server. Thus, part of the application logic resides on the server, in stored procedures, views, aliases, according to the idea that the data to be actually transferred should be of minimum size - complete elimination of redundant/nonnecessary information. Both advantages of Oracle as a preferment DBMS, and Delphi, as a high level programming environment and language were taken into consideration, thus, the client had to take over a big amount of the application logic, the main reason being also efficiency. For fetching and storing data at user-input level, mainly standard abstract Visual Controls Library objects provided by Borland Delphi are used.
Nevertheless, because of the special characteristics of the project and mainly of the big problems caused by the huge amount of data to be manipulated, and of the lengthy operations needed to be performed with this data, not always could the implementation follow standard techniques, and new components had to be developed and different non-standard techniques to be employed. Several versions of the client application, and of the system as a whole, were issued, both for testing and for complying with different needs. Lately a second version of the system was developed based on Multi-tier technology (ASP, CGI, ActiveX technology) for deploying the whole system on Web. This is a version which eliminates the need for additional drivers on client machine (namely the need for the Oracle client software) and allows users to access the system (with proper security privileges) from any site on the internet.
System technical specs (year 1998)
Database Server:
• Pentium Pro Dual - 2x200, 128MB RAM, 16 GB HDD
• Windows NT 4.0, sp6
• Oracle 7.3
Application Server :
• Pentium Pro Dual - 2x200, 128MB RAM
• Windows NT 4.0, sp6
• Oracle Client (SQL *Net)
• Internet Information Server with ASP and ADO
Clients:
Standard:
• Windows 32 (NT-98), minimum 16 MB RAM, minimum 500 MB HDD
• Oracle Client (SQL *Net)
• Only for export operations: MS Excel installed.
Web clients:
• Browser with ActiveX support
• Internet connection (with or without PROXY)
Security:
• user/password authentication.
• data encryption
For web access:
• Coded data transfer through HTTP (2nd layer of data encryption)
• Initial authentication (two-levels)
• Two levels of tracking / restricting access: at HTTP server requests (IP addresses, site users) and user-level (application users, defined and managed by the TSDB system). |
|
