Difference between revisions of "Redistribution and managing dependencies"
Line 118: | Line 118: | ||
|- | |- | ||
| || xercesImpl.jar || '''Yes''' || Core third party file || | | || xercesImpl.jar || '''Yes''' || Core third party file || | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
|- | |- |
Revision as of 16:08, 24 December 2014
Redistribution list
Folder | File | Required | Needed for | Depends on |
---|---|---|---|---|
home | Yes | Root folder | ||
etl.exe | No | Windows executable | ||
etl.app | No | Mac application bundle | ||
etl.sh | No | Unix/Linux shell script | ||
etlappstart.properties | No | JVM properties | ||
etl.jar | Yes | Main executable jar | ||
home\config | No | Default configuration folder | ||
etl.properties | No | Configuration properties | ||
etl_config.xml | No | Default ETL configuration file | ||
log4j.properties | No | Log configuration | ||
home\license | No | Folder for licenses | ||
*.lic | No | Licenses (ETL framework will also look at license.jar under home\lib folder) |
| |
home\data | No | Default folder for data | ||
home\data\scenario | No | Default folder for ETL scenarios | ||
home\data\schema | No | Default folder for XLT schema files | ||
dataset2webrowset.xsl | No | XSL schema file for Toolseverse XML dataset to WebRowSet XML transformation | ||
webrowset2dataset.xsl | No | XSL schema file for WebRowSet XML to Toolseverse XML dataset transformation | ||
xmldataset.xsd | No | Schema file for Toolsverse XML dataset file format | ||
home\lib | No | Folder for core and third party jar files. Files from this folder automatically added to the classpath | ||
antlr-3.0.1.jar | Yes | Core third party file | ||
antlr.jar | Yes | Core third party file | ||
commons-beanutils-1.9.1.jar | Yes | Core third party file | ||
commons-codec-1.6.jar | Yes | Core third party file | ||
commons-collections4-4.0.jar | Yes | Core third party file | ||
commons-dbcp-1.4.jar | Yes | Core third party file | ||
commons-logging-1.1.3.jar | Yes | Core third party file | ||
commons-math3-3.0.jar | Yes | Core third party file | ||
commons-net-2.0.jar | Yes | Core third party file | ||
commons-pool-1.5.4.jar | Yes | Core third party file | ||
dom4j-1.6.1.jar | Yes | Core third party file | ||
httpclient-4.3.3.jar | Yes | Core third party file | ||
httpclient-cache-4.3.3.jar | Yes | Core third party file | ||
httpcore-4.3.2.jar | Yes | Core third party file | ||
httpmime-4.3.3.jar | Yes | Core third party file | ||
javassist.jar | Yes | Core third party file | ||
json-io.jar | Yes | Core third party file | ||
log4j-1.2.16.jar | Yes | Core third party file | ||
rsyntaxtextarea.jar | Yes | Core third party file | ||
RSyntaxTextArea.License.txt | Yes | Core third party file | ||
saxon9he.jar | Yes | Core third party file | ||
xercesImpl.jar | Yes | Core third party file | ||
toolsverse-core.jar | Yes | Toolsverse core file | ||
toolsverse-etl-common.jar | Yes | Toolsverse core file | ||
toolsverse-etl-connector-sql.jar | Yes | Toolsverse core file | ||
toolsverse-etl-connector.jar | Yes | Toolsverse core file | ||
toolsverse-etl-core.jar | Yes | Toolsverse core file | ||
toolsverse-etl-driver.jar | Yes | Toolsverse core file | ||
toolsverse-etl-metadata.jar | Yes | Toolsverse core file | ||
toolsverse-io.jar | Yes | Toolsverse core file | ||
toolsverse-license.jar | Yes | Toolsverse core file | ||
toolsverse-mvc.jar | Yes | Toolsverse core file | ||
toolsverse-security.jar | Yes | Toolsverse core file | ||
toolsverse-service.jar | Yes | Toolsverse core file | ||
toolsverse-storage.jar | Yes | Toolsverse core file | ||
toolsverse-updater.jar | Yes | Toolsverse core file |
| |
derby.jar | No | Derby DB JDBC driver | ||
iText-4.2.0-com.itextpdf.jar | No | Read and write PDF files | ||
javax.json-1.0.4.jar | No | Read and write JSON files | ||
javax.mail.jar | No | Send emails | ||
jsch-0.1.43.jar | No | SFTP client | ||
poi-3.8-20120326.jar | No | Read and write Excel files | ||
poi-ooxml-3.8-20120326.jar | No | Read and write Excel files | ||
poi-ooxml-schemas-3.8-20120326.jar | No | Read and write Excel files | ||
servlet-api.jar | No | Required if you are going to call Web services | ||
socks.jar | No | Socks proxy used together with FTP | ||
home\plugin | No | Folder for plugins. In other to use files in this folder you must have a valid not expired license | ||
toolsverse-etl-connector-excel.jar | No | Required if you are going to read and write Excel files | toolsverse-etl-connector-json.jar | |
toolsverse-etl-connector-html.jar | No | Required if you are going to read and write HTML files | ||
toolsverse-etl-connector-json.jar | No | Required if you are going to read and write JSON files | ||
toolsverse-etl-connector-pdf.jar | No | Required if you are going to read and write PDF files | ||
toolsverse-etl-connector-text.jar | No | Required if you are going to read and write CVS and fixed-length files | toolsverse-etl-connector-json.jar, toolsverse-etl-connector-xml.jar | |
toolsverse-etl-connector-xml.jar | No | Required if you are going to read and write XML files | toolsverse-etl-connector-json.jar | |
toolsverse-etl-db2.jar | No | Required if you are going to work with DB2 databases | ||
toolsverse-etl-informix.jar | No | Required if you are going to work with Informix databases | ||
toolsverse-etl-oracle.jar | No | Required if you are going to work with Oracle databases | ||
toolsverse-etl-mysql.jar | No | Required if you are going to work with MySQL databases | ||
toolsverse-etl-postgres.jar | No | Required if you are going to work with Postgres databases | ||
toolsverse-etl-sybase.jar | No | Required if you are going to work with Sybase ASE databases | ||
home\jdbc | No | Default folder for JDBC drivers |
|
Example
home etl.exe etl.app etl.jar etl.sh etlappstart.properties config etl.properties etl_config.xml log4j.properties license *.lic data scenario schema dataset2webrowset.xsl webrowset2dataset.xsl xmldataset.xsd lib antlr-3.0.1.jar antlr.jar commons-beanutils-1.9.1.jar commons-codec-1.6.jar commons-collections4-4.0.jar commons-dbcp-1.4.jar commons-logging-1.1.3.jar commons-math3-3.0.jar commons-net-2.0.jar commons-pool-1.5.4.jar derby.jar dom4j-1.6.1.jar httpclient-4.3.3.jar httpclient-cache-4.3.3.jar httpcore-4.3.2.jar httpmime-4.3.3.jar iText-4.2.0-com.itextpdf.jar javassist.jar javax.json-1.0.4.jar javax.mail.jar jsch-0.1.43.jar json-io.jar log4j-1.2.16.jar poi-3.8-20120326.jar poi-ooxml-3.8-20120326.jar poi-ooxml-schemas-3.8-20120326.jar rsyntaxtextarea.jar RSyntaxTextArea.License.txt saxon9he.jar servlet-api.jar socks.jar toolsverse-core.jar toolsverse-etl-aliasreader.jar toolsverse-etl-common.jar toolsverse-etl-connector-sql.jar toolsverse-etl-connector.jar toolsverse-etl-core.jar toolsverse-etl-driver.jar toolsverse-etl-metadata.jar toolsverse-io.jar toolsverse-license.jar toolsverse-mvc.jar toolsverse-security.jar toolsverse-service.jar toolsverse-storage.jar toolsverse-updater.jar xercesImpl.jar xml-apis.jar resolver.jar serializer.jar xmlbeans-2.3.0.jar plugin toolsverse-etl-connector-excel.jar toolsverse-etl-connector-html.jar toolsverse-etl-connector-json.jar toolsverse-etl-connector-pdf.jar toolsverse-etl-connector-text.jar toolsverse-etl-connector-xml.jar toolsverse-etl-db2.jar toolsverse-etl-informix.jar toolsverse-etl-mysql.jar toolsverse-etl-oracle.jar toolsverse-etl-postgres.jar toolsverse-etl-sqlserver.jar
Managing dependencies, files and folders
Files and folders marked as required must exist in order for framework to work. Others are optional.
Framework is tested with the particular versions of the third party libraries (see redistribution list). It is likely framework will work with more recent versions than with older versions.
Jars under lib and plugin folders are automatically added to the classpath on start-up if there is an available license. Only commercial components of the ETL framework require license.
You can reduce number of folders ETL framework depends on:
- Move all files from the plugin folder to lib.
- Package existing licenses (*.lic files) in the license folder into jar file and copy jar into lib folder. Name of the jar file doesn't matter but lic files must be located in the com/toolsverse/license folder inside jar.
- Files in the config folder and config folder itself are not required. You can pass properties to the framework using -Dproperty_name=value.
Framework uses HOME/data/ folder for temporary files, scripts and data files. If folder does not exist it is automatically created.
By default framework is looking for ETL scenarios in the HOME/data/scenario folder. If you want to change base folder for scenarios use -D property scenario.path.
Look at the src/com/toolsverse/config/SystemConfig.java how ETL frameworks uses system folders.
Look at the src/com/toolsverse/etl/core/config/EtlConfig.java how ETL frameworks uses ETL specific folders.