public interface Block extends LocationProvider
Source
and
Destination
classes.Modifier and Type | Interface and Description |
---|---|
static class |
Block.BlockType
Possible block types
|
Modifier and Type | Method and Description |
---|---|
void |
enableMetrics(boolean value)
Enable/disable capturing metrics.
|
ListHashMap<String,Task> |
getBeforeEtlTasks()
Gets the tasks which are executed before etl framework starts processing
the
DataSet object associated with the particular scenario
block but after initializing is done. |
Block.BlockType |
getBlockType()
Gets the {#link BlockType}.
|
Connection |
getConnection()
Gets the actual JDBC connection associated with the scenario block.
|
String |
getConnectionName()
Gets the name of the connection associated with the scenario block.
|
DataSet |
getDataSet()
Gets the
DataSet object associated with the scenario block. |
String |
getDriverClassName()
Gets the driver class name.
|
String |
getHighWatermarkField()
Gets the high watermark field.
|
ListHashMap<String,Task> |
getInlineTasks()
Gets the tasks which are executed for each row of the
DataSet object associated with the scenario block. |
TypedKeyValue<Point,Point> |
getLocation(String key)
Gets the location of the collection artifact such as tasks, variables, etc.
|
Map<String,TypedKeyValue<Point,Point>> |
getLocations()
The the locations of the collection artifacts such as tasks, variables, etc.
|
String |
getName()
Gets name of the block.
|
ListHashMap<String,Task> |
getOnExceptionTasks()
Gets the tasks which are executed when there is an exception during processing of the etl block.
|
String |
getOriginalName()
Gets the original block name.
|
ListHashMap<String,Task> |
getPostTasks()
Gets the tasks which are executed after etl framework finished with the
processing of the particular scenario block.
|
ListHashMap<String,Task> |
getTasks()
Gets the tasks which are executed before etl framework starts processing
of the particular scenario block.
|
Variable |
getVariable(String... name)
Gets the variable by name.
|
ListHashMap<String,Variable> |
getVariables()
Gets the variables.
|
boolean |
isEmpty()
Checks if
DataSet object associated with the scenario block
is empty. |
boolean |
isEnabled()
Checks if scenario block is enabled.
|
boolean |
isEncoded()
Checks if encoded flag set for the
DataSet object associated
with the scenario block. |
boolean |
isMetricsEnabled()
Checks if capturing metrics enabled.
|
boolean |
isParallel()
Checks if code for the particular scenario block can be executed in the
separate thread.
|
boolean |
noConnection()
Check if no database connection should be associated with the block.
|
void |
setBeforeEtlTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed before etl framework starts processing
the
DataSet object associated with the particular scenario
block but after initializing is done. |
void |
setConnection(Connection value)
Sets the actual JDBC connection associated with the scenario block.
|
void |
setConnectionName(String value)
Sets the name of the connection associated with the scenario block.
|
void |
setDataSet(DataSet value,
boolean adjust)
Sets the
DataSet object associated with the scenario block. |
void |
setDriverClassName(String value)
Sets the driver class name.
|
void |
setEnabled(boolean value)
Sets the enabled property of the scenario block.
|
void |
setEncoded(boolean value)
Sets the encoded flag for the
DataSet object associated with
the scenario block. |
void |
setInlineTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed for each row of the
DataSet object associated with the scenario block. |
void |
setIsEmpty(boolean value)
Sets the state of the field
isEmpty for the scenario block
regardless of the actual state of the DataSet object. |
void |
setLocation(boolean doSet,
String key,
org.w3c.dom.Node treesNode)
Set the location for the artifacts such as tasks, variables, etc.
|
void |
setLocation(boolean doSet,
String key,
TypedKeyValue<Point,Point> location)
Set the location for the artifacts such as tasks, variables, etc.
|
void |
setName(String value)
Sets the name of the block.
|
void |
setOnExceptionTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed when there is an exception during processing of the etl block.
|
void |
setOriginalName(String name)
Sets the original block name.
|
void |
setParallel(boolean value)
Sets the the property isParallel for the scenario block.
|
void |
setPostTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed after etl framework finished with the
processing of the particular scenario block.
|
void |
setTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed before etl framework starts processing
of the particular scenario block.
|
void |
setVariables(ListHashMap<String,Variable> value)
Sets the variables for the block.
|
getEndLocation, getStartLocation
void enableMetrics(boolean value)
value
- the valueListHashMap<String,Task> getBeforeEtlTasks()
DataSet
object associated with the particular scenario
block but after initializing is done. For example: after metadata are
collected.ListHashMap
objectBlock.BlockType getBlockType()
Connection getConnection()
String getConnectionName()
Source
or Destination
Example of named connection in the scenario file:
<destination>
<name>image</name>
<source>image</source>
<load>
<connection>mssql</connection>
<driver>com.toolsverse.etl.driver.mysql.MsSqlDriver</driver>
</load>
</destination>
DataSet getDataSet()
DataSet
object associated with the scenario block.String getDriverClassName()
String getHighWatermarkField()
ListHashMap<String,Task> getInlineTasks()
DataSet
object associated with the scenario block.ListHashMap
objectTypedKeyValue<Point,Point> getLocation(String key)
key
- the artifact type nameMap<String,TypedKeyValue<Point,Point>> getLocations()
String getName()
ListHashMap<String,Task> getOnExceptionTasks()
ListHashMap
objectListHashMap<String,Task> getPostTasks()
ListHashMap
objectListHashMap<String,Task> getTasks()
ListHashMap
objectVariable getVariable(String... name)
name
- the name of the variableListHashMap<String,Variable> getVariables()
boolean isEmpty()
DataSet
object associated with the scenario block
is empty. Scenario block can be configured to be treated as empty
regardless of the actual state of the DataSet
object.boolean isEnabled()
boolean isEncoded()
DataSet
object associated
with the scenario block. Usually only Clob and Blob fields can be encoded. ETL
framework uses open source implementation of the Base64 encoding.boolean isMetricsEnabled()
boolean isParallel()
boolean noConnection()
void setBeforeEtlTasks(ListHashMap<String,Task> value)
DataSet
object associated with the particular scenario
block but after initializing is done. For example after metadata are
collected. Tasks get instantiated by parsing scenario file.value
- the new before etl tasksvoid setConnection(Connection value)
value
- the new connectionvoid setConnectionName(String value)
value
- the new name of the connectionvoid setDataSet(DataSet value, boolean adjust)
DataSet
object associated with the scenario block.value
- the new data setadjust
- if true adjust data set fieldsvoid setDriverClassName(String value)
value
- the new driver class namevoid setEnabled(boolean value)
true
.
Example of the manually set enabled flag in the scenario file:
<source enabled="No">
<name>test</name>
<extract>
<sql>select * from test</sql>
</extract>
</source>
value
- the new enabledvoid setEncoded(boolean value)
DataSet
object associated with
the scenario block. By default all DataSet
objects which
contain Clob or Blob fields are encoded and compressed using zip
algorithm.
Example of the manually set encoded flag in the scenario file:
<source encode="No">
<name>clobs</name>
<extract>
<sql>select * from clobs</sql>
</extract>
</source>
value
- the new encodedvoid setInlineTasks(ListHashMap<String,Task> value)
DataSet
object associated with the scenario block. Inline
tasks get instantiated by parsing scenario file.
Example of the inline task in the scenario file:
<tasks>
<task>
<name>replace</name>
value
- the value
<class>com.toolsverse.etl.core.task.common.RegexpTransformator</class>
<variables>
<FIELD value="DESCRIPTION" />
<REGEXP value="Substitution" />
<REPLACE value="something" />
</variables>
</task>
</tasks>
void setIsEmpty(boolean value)
isEmpty
for the scenario block
regardless of the actual state of the DataSet
object. In the
majority of cases etl framework can automatically set this flag however
it doesn't hurt to do it manually.
Example of the always empty scenario block in the scenario file:
<source empty="Yes">
<name>init</name>
<extract>
<tasks>
<task>
<name>prepare</name>
<class>com.toolsverse.etl.core.task.sql.SqlTask</class>
<sql>drop table tmp_cp</sql>
<onexception action="ignore"/>
</task>
</tasks>
</extract>
</source>
value
- the new isEmptyvoid setLocation(boolean doSet, String key, org.w3c.dom.Node treesNode)
doSet
- if false do not add location info for the artifactkey
- the artifact typetreesNode
- the DOM node which holds location informationvoid setLocation(boolean doSet, String key, TypedKeyValue<Point,Point> location)
doSet
- if false do not add location info for the artifactkey
- the artifact typelocation
- the locationvoid setName(String value)
Example of the setting of the Destination name in scenario file:
<destination>
<name>image</name>
</destination>
Example of the setting of the Source name in scenario file:
<source>
<name>image</name>
</source>
value
- The new namevoid setOnExceptionTasks(ListHashMap<String,Task> value)
Example of the post task in the scenario file:
<tasks>
<task scope="onexception">
<name>delete</name>
<class>com.toolsverse.etl.core.task.common.FileManagerTask</class>
<variables>
<COMMAND value="delete" />
<SOURCE_TYPE value="file" />
<FILES value="*.dat" />
</variables>
</task>
</tasks>
value
- the new post tasksvoid setParallel(boolean value)
true
the code for this block can be executed in the separate
thread. The default value is false
.
Example of the parallel scenario block in the scenario file:
<source parallel="Yes">
<name>test</name>
<extract>
<sql>select * from test</sql>
</extract>
</source>
value
- the new parallelvoid setPostTasks(ListHashMap<String,Task> value)
Example of the post task in the scenario file:
<tasks>
<task scope="after">
<name>delete</name>
<class>com.toolsverse.etl.core.task.common.FileManagerTask</class>
<variables>
<COMMAND value="delete" />
<SOURCE_TYPE value="file" />
<FILES value="*.dat" />
</variables>
</task>
</tasks>
value
- the new post tasksvoid setTasks(ListHashMap<String,Task> value)
Example of the task in the scenario file:
<tasks>
<task>
<name>copy</name>
<connection>dest</connection>
<class>com.toolsverse.etl.core.task.oracle.OracleCopyTask</class>
<sql>
insert temp USING
select * from some_table
</sql>
</task>
</tasks>
value
- the new tasksvoid setVariables(ListHashMap<String,Variable> value)
Example of the variables in the scenario file:
<variables>
<STEP_NUM function="getPk" />
<CONFIG_NUM function="getFk" />
<USER function="getUser" />
<ID include="No" />
</variables>
value
- The new variablesString getOriginalName()
void setOriginalName(String name)
name
- String the new original nameCopyright © 2010-2020 Toolsverse. All Rights Reserved.