Product Overview

Content of this page:

1. Commercial issues

Who commercialises ETL4ALL?

ETL4ALL is commercialised by IKAN Software NV (www.ikan.be).

Are there different product versions available for different user needs?

ETL4All is available in different versions, with different license fees. The simplest configuration is the stand-alone configuration. Here, all processes run on a single machine. The data that are accessed need to be present on the LAN or WAN that ETL4ALL is installed on. When data are distributed over separate sites, connected through the internet, or when several users are co-operating, the client server configuration is more suitable. This version allows data access to any kind of data source or target, with no location restrictions. It allows both to scale ETL4ALL's transformation power and to avoid system failure by clustering multiple ETL4ALL server instances.

2. General topics

What are the key characteristics of the design of ETL4ALL?

The key characteristics are :

User friendliness. ETL4ALL is designed for ease of use. Every end-user should be able to complete the whole process from source data to sink data respectively report.
Maximal flexibility at the data layer.
We offer access to virtually every source containing structured data (see the section on data extraction), or are capable of building the access very quickly.
Distributed environment.
All different parts of the product (data layer / ETL4ALL server / client application) can run on separate machines, connected through the Internet.
Maximal scalability.
We can add and remove data sources and transformation engines on the fly.
Standards based.
ETL4ALL is built around standards like XML, XSLT, JDBC and SOAP. This allows the product to take full profit of the rapid development in these areas, and results in a small learning curve.
ETL4ALL is a highly modular product that uses some libraries for implementing SOAP, processing XML and transforming XSLT. For a standard installation we recommend the use of third-party (public domain) libraries. Next to the fact that these libraries are for free, they are being optimised at a high pace.

How secure is ETL4ALL?

Standard, the distributed ETL4ALL version has password protection for the access from the ETL4ALL client application to the ETL4ALL server. The data access objects are in practice servlets running on an application server. Securing access to these is a matter of configuring your web application.

What about scalability?

The distributed ETL4ALL application is extremely scalable. It allows adding data access objects, servers and clients on the fly. In distributed configuration, each of these data access objects can run on a different machine, as long as it is network accessible. In standalone mode, adding data access objects involves restarting ETL4ALL.

Can you tell something about the performance of ETL4ALL?

To offer maximum flexibility at the data layer, ETL4ALL converts all incoming data to XML and processes the resulting stream using a transformation engine (XSLT). This results in a performance that is generally less than tools that avoid this conversion. However, current tests do not reveal performance problems.

Obviously, it is possible to execute multiple jobs concurrently. Multithreaded operation, parallel code execution and load balancing are standard features.

What about training?

Through the use of common standards (XML, XSLT) we try to avoid confronting the user to things he/she is unfamiliar to. Obviously, IKAN Software NV also provides post-sales support. Check the web site for details.

3. Product architecture

What are the components of ETL4ALL?

ETL4ALL consists of 3 different components:

Data access objects. Data access objects are servlets running in an application server's servlet container. They provide the conversion from data in random data sources to XML data that will be used by the ETL4ALL server. The data access objects are accessible through a set of standardized calls.
The ETL4ALL server. The ETL4ALL server acts as ETL4ALL's main control centre. It is a SOAP server, running in an application server. It is the central access point for all other components. It is responsible for access control, load balancing, etc. The server embeds a transformation engine object. A transformation engine is built upon a trAX compliant XSLT engine, and is used by the ETL4ALL server when data transformations are required.
ETL4ALL client applications. An ETL4ALL client application can be any kind of application, capable of talking SOAP to the ETL4ALL server. Our own ETL4ALL client provides an extremely user-friendly interface to compose transformations and to debug them.

Note: in the stand-alone version of ETL4ALL, all these components are obviously integrated into a single program.

What platforms are supported?

ETL4ALL is a full java application, running on any platform for which a java virtual machine is available. Evidently, this limitation does not apply for the data sources. These simply have to be accessible (e.g. using JDBC) from a platform for which a java virtual machine exists. As a consequence, access to for instance mainframe data sources is no problem at all.

Is there a recovery mechanism when a system crash occurs during ETL processing?

Recovery is not provided version 2.4.0. When a transformation fails, the original data remain unchanged.

Is error reporting provided?

Currently, straightforward logging is provided. This logging is configurable in the sense that 4 levels of logging details are provided. At the most detailed level this results in storing information about the source, target, processing time and result of every transformation. At the least detailed level, we store only possible failures.

4. Data extraction and loading

What kind of data types can be processed using ETL4ALL?

We offer access to all relational databases through JDBC, as well as to XML databases. A by no means complete list of supported sources is the following:

Adabas
Adabas/C
ALLBASE/SQL
C-ISAM (Informix)
C-ISAM (Microfocus)
C-ISAM ACCUCORP
CA-Datacom
CA-IDMS/DB
CA-IDMS/SQL
Cloudscape
D-ISAM
DB2
DB2/400
DBASE
DBMS
EJB
ENSCRIBE
Essbase
Excel
FOCUS Data Access
Foxpro
IDS-II
ImageSQL
IMS
InfoMan
Informix
Ingres
Interplex
ISAM
KSAM
Lotus Notes
Microsoft Access
Microsoft OLAP Services
Microsoft SQL Server
Millennium
Model 204
MUMPS (Digital Standard MUMPS)
MySQL
NOMAD
NonStop SQL
Nucleus
Omnidex
OpenIngres
Oracle
PROGRESS
QSAM
Rdb
Red Brick
RMS
SAP BW
SAP R/3
SQL/DS
SUPRA
Sybase
Sybase/IQ
System 2000
Teradata
TOTAL
TurboIMAGE
Unisys DMS 1100/2200
UNISYS DMS II
UniVerse
VSAM
XML
CSV Files
Xindice

How are new data brought under control of ETL4ALL?

ETL4ALL offers administrative functionality to inspect relational databases and drag and drop interesting tables into the system. Any type of file is added to ETL4ALL by simply dragging it from a file explorer.

Does ETL4ALL support real-time data processing?

Both the distributed and stand-alone configuration allow the retrieval and inspection of data obtained after transformation in exactly the same way as if these data existed in a file or a database. In this sense, ETL4ALL supports real-time data processing. Furthermore, the distributed configuration supports real-time data updating, in the sense that adding a new file at a predefined location can act as a trigger to start a chain of transformations and data updates. The ETL4ALL stand-alone configuration does not support real-time data processing.

How are data moved in the ETL4ALL system?

Data are moved over the Internet using a HTTP POST request. In the ETL4ALL design, we have taken the necessary precautions in order to make the use of for instance message-oriented middleware a straightforward modification. For transferring data in and out databases, JDBC is used.

Does ETL4ALL support huge transformations?

ETL4ALL splits its source data into parts that are small enough to handle. As a consequence any kind of transformation can be theoretically handled.