public class Source extends OnException implements Block, ReadTracker
Source
is a one of the two building blocks of the ETL scenario:
etl process extracts data from the Sources and loads into the Destinations.
It implements Block
interface and adds some methods and members
specific for the Source. There are can be multiple sources in the scenario.
In some cases the code for the source can be executed in the separate thread.
Example of source declaration in the scenario file:
<source>
<name>image</name>
<extract>
<sql>select * from image</sql>
</extract>
</source>
Block.BlockType
Modifier and Type | Field and Description |
---|---|
static String |
BEFORE_SQL_ATTR
BEFORE_SQL_ATTR.
|
static String |
COND_CONNECTION_ATTR
COND_CONNECTION_ATTR.
|
static String |
COND_lANG_ATTR
COND_lANG_ATTR.
|
static String |
CONDITION_ATTR
CONDITION_ATTR.
|
static String |
CONNECTION_ATTR
CONNECTION_ATTR.
|
static String |
EMPTY_ATTR
EMPTY_ATTR.
|
static String |
ENABLED_ATTR
ENABLED_ATTR.
|
static String |
ENCODE_ATTR
ENCODE_ATTR.
|
static String |
ETL_DRIVER_ATTR
ETL_DRIVER_ATTR.
|
static String |
EXCLUDE_ATTR
EXCLUDE_ATTR.
|
static String |
FETCH_SIZE_ATTR
FETCH_SIZE_ATTR.
|
static String |
IGNORE_EMPTY_ATTR
IGNORE_EMPTY_ATTR.
|
static String |
IGNORE_NO_RECORDS_ATTR
IGNORE_NO_RECORDS_ATTR.
|
static String |
INCLUDE_ATTR
INCLUCE_ATTR.
|
static String |
INDEPENDENT_ATTR
INDEPENDENT_ATTR.
|
static String |
KEY_FIELDS_ATTR
KEY_FIELDS_ATTR.
|
static String |
LIMIT_ATTR
LIMIT_ATTR.
|
static String |
LINKED_SOURCE_ATTR
LINKED_SOURCE_ATTR.
|
static String |
MANDATORY_ATTR
MANDATORY_ATTR.
|
static String |
MATCH_ATTR
MATCH_ATTR.
|
static String |
MAX_FILES_ATTR
MAX_FILS_ATTR.
|
static String |
METRICS_ENABLED_ATTR
The METRICS_ENABLED_ATTR.
|
static String |
NAME_ATTR
NAME_ATTR.
|
static String |
NEW_CONNECTION_ATTR
NEW_CONNECTION_ATTR.
|
static String |
NEW_CONNECTION_SOURCE_QUERY_ATTR
NEW_CONNECTION_SOURCE_QUERY_ATTR.
|
static String |
NO_CONNECTION_ATTR
NO_CONNECTION_ATTR.
|
static String |
OBJECT_NAME_ATTR
OBJECT_NAME_ATTR.
|
static String |
ON_EXCEPTION_ACTION_ATTR
ON_EXCEPTION_ACTION_ATTR.
|
static String |
ON_EXCEPTION_MASK_ATTR
ON_EXCEPTION_MASK_ATTR.
|
static String |
ON_PERSIST_ATTR
ON_PERSIST_ATTR.
|
static String |
ON_POPULATE_ATTR
ON_POPULATE_ATTR.
|
static String |
OWNER_ATTR
OWNER_ATTR.
|
static String |
PARALLEL_ATTR
PARALLEL_ATTR.
|
static String |
READER_ATTR
READER_ATTR.
|
static String |
SOURCE_ATTR
SOURCE_ATTR.
|
static String |
SOURCE_SQL_ATTR
SOURCE_SQL_ATTR.
|
static String |
SQL_ATTR
SQL_ATTR.
|
static String |
SQL_STATEMENT_OUTPUT_PARAM_INDEEX_ATTR
OUTPUT PARAMETERS INDEX ATTR.
|
static String |
SQL_STATEMENT_PARAM_INDEX_ATTR
OUTPUT PARAMETER INDEX FOR CALLABLE STATEMENT ATTR.
|
static String |
SQL_STATEMENT_TYPE_ATTR
EXTRACT SQL STATEMENT TYPE ATTR.
|
static String |
TRACKING_ENABLED_ATTR
The TRACKING_ENABLED_ATTR.
|
static String |
UPDATE_KEY_ATTR
UPDATE_KEY_ATTR.
|
static String |
USING_ATTR
USING_ATTR.
|
static String |
WILDCARD_PARALLEL_ATTR
WILDCARD_PARALLEL_ATTR.
|
static String |
WRITER_ATTR
WRITER_ATTR.
|
ON_EXCEPTION, ON_EXCEPTION_CONTINUE, ON_EXCEPTION_CONTINUE_STR, ON_EXCEPTION_IGNORE, ON_EXCEPTION_IGNORE_STR, ON_EXCEPTION_MERGE, ON_EXCEPTION_MERGE_STR, ON_EXCEPTION_RAISE, ON_EXCEPTION_RAISE_KEY, ON_EXCEPTION_RAISE_STR, ON_EXCEPTION_TYPE, ON_PARSE_EXCEPTION, ON_PARSE_EXCEPTION_STR
Constructor and Description |
---|
Source()
Instantiates a new source.
|
Modifier and Type | Method and Description |
---|---|
void |
addVariable(Variable var)
Adds the variable.
|
void |
decUsageCounter()
Decrements usage counter.
|
void |
enableMetrics(boolean value)
Enable/disable capturing metrics.
|
ListHashMap<String,Task> |
getBeforeEtlTasks()
Gets the tasks which are executed before etl framework starts processing
the
DataSet object associated with the particular scenario
block but after initializing is done. |
String |
getBeforePrepareSourceQuery()
Gets the JavaScript code to execute before prepare source query.
|
Block.BlockType |
getBlockType()
Gets the {#link BlockType}.
|
StreamingCommitPolicy |
getCommitPolicy()
Gets the commit policy.
|
Connection |
getConnection()
Gets the actual JDBC connection associated with the scenario block.
|
String |
getConnectionName()
Gets the name of the connection associated with the scenario block.
|
String |
getDataReaderClassName()
Gets the data reader class name.
|
Map<String,String> |
getDataReaderParams()
Gets the data reader params.
|
DataSet |
getDataSet()
Gets the
DataSet object associated with the scenario block. |
String |
getDataWriterClassName()
Gets the data writer class name.
|
Map<String,String> |
getDataWriterParams()
Gets the data writer params.
|
String |
getDefaultConnectionName()
Gets the default connection name.
|
Map<String,Object> |
getDefaultValues(Scenario scenario)
Gets the default values for the model used to edit the source.
|
String |
getDriverClassName()
Gets the driver class name.
|
Point |
getEndLocation()
Gets the end location.
|
String |
getExclude()
Gets the comma separated list of the excluded table for the wildcard
matching.
|
int |
getFetchSize()
Gets the fetch size.
|
Map<String,String> |
getFiles()
Gets the names of the extracted files.
|
String |
getHighWatermarkField()
Gets the high watermark field.
|
String |
getIdForException()
Gets the id for exception.
|
String |
getInclude()
Gets the comma separated list of the included table for the wildcard
matching.
|
ListHashMap<String,Task> |
getInlineTasks()
Gets the tasks which are executed for each row of the
DataSet object associated with the scenario block. |
String |
getKeyFields()
Gets the name of the key fields.
|
String |
getKeyName()
Gets the key name.
|
String |
getLinkedSourceName()
Gets the name of the linked source.
|
TypedKeyValue<Point,Point> |
getLocation(String key)
Gets the location of the collection artifact such as tasks, variables, etc.
|
Map<String,TypedKeyValue<Point,Point>> |
getLocations()
The the locations of the collection artifacts such as tasks, variables, etc.
|
String |
getMatch()
Gets the match.
|
int |
getMaxFiles()
Gets the max files to process when wildcard processing is nabled.
|
int |
getMaxRows()
Gets the maximum number of rows to extract.
|
String |
getName()
Gets the name.
|
String |
getObjectName()
Gets the name of the object.
|
ListHashMap<String,Task> |
getOnExceptionTasks()
Gets the tasks which are executed when there is an exception during processing of the etl block.
|
int |
getOnPersistDataSet()
Gets the action on "persist data set" event.
|
int |
getOnPersistDataSet(String action)
Gets the action on "persist data set" from then given action name.
|
String |
getOnPersistDataSetAction()
Gets the on persist data set action.
|
int |
getOnPopulateDataSet()
Gets the action on "populate data set" event.
|
int |
getOnPopulateDataSet(String action)
Gets the action on "populate data set" event from the given action name.
|
String |
getOnPopulateDataSetAction()
Gets the on populate data set action.
|
String |
getOriginalName()
Gets the original block name.
|
String |
getOutputParamIndexes()
Gets the comma delimited list of of indexes for the output parameters to
get a result set from callable statement.
|
String |
getOwnerName()
Gets the owner name.
|
List<Integer> |
getParamIndexes()
Gets the array of indexes for the output parameters to get a result set
from callable statement.
|
ListHashMap<String,Task> |
getPostTasks()
Gets the tasks which are executed after etl framework finished with the
processing of the particular scenario block.
|
String |
getSql()
Gets the sql.
|
Point |
getStartLocation()
Gets the start location.
|
SqlStatementType |
getStatementType()
Gets the statement type.
|
ListHashMap<String,Task> |
getTasks()
Gets the tasks which are executed before etl framework starts processing
of the particular scenario block.
|
int |
getUsageCounter()
Gets the usage counter.
|
String |
getUsing()
Gets the "using".
|
Variable |
getVariable(String... name)
Gets the variable by name.
|
ListHashMap<String,Variable> |
getVariables()
Gets the variables.
|
void |
incUsageCounter()
Increments usage counter.
|
boolean |
isEmpty()
Checks if
DataSet object associated with the scenario block
is empty. |
boolean |
isEnabled()
Checks if scenario block is enabled.
|
boolean |
isEncoded()
Checks if encoded flag set for the
DataSet object associated
with the scenario block. |
boolean |
isIgnoreEmpty()
Checks if ignore empty source flag is set.
|
boolean |
isIgnoreNoRecords()
Checks if ignore empty no records in the source flag is set.
|
Boolean |
isIndependent()
Checks if source is independent.
|
boolean |
isMandatory()
Checks if source is mandatory.
|
boolean |
isMetricsEnabled()
Checks if capturing metrics enabled.
|
boolean |
isNewConnection()
Checks if new source connection should be created for each transformation.
|
boolean |
isNewConnectionSourceQuery()
Checks if new connection should be created for each SQL in source query.
|
boolean |
isParallel()
Checks if code for the particular scenario block can be executed in the
separate thread.
|
boolean |
isStub()
Checks if source is a stub.
|
boolean |
isTrackingEnabled()
Checks if tracking is allowed.
|
boolean |
isUpdateIndexOnChange()
Checks if index should be updated automatically.
|
boolean |
isWildcardParallel()
Checks if parallel processing of wildcard sources is enabled.
|
boolean |
noConnection()
Check if no database connection should be associated with the block.
|
void |
setBeforeEtlTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed before etl framework starts processing
the
DataSet object associated with the particular scenario
block but after initializing is done. |
void |
setBeforePrepareSourceQuery(String beforePrepareSourceQuery)
Sets the JavaScript code to execute before prepare source query.
|
void |
setBlockType(Block.BlockType value)
Sets the block type.
|
void |
setCommitPolicy(StreamingCommitPolicy commitPolicy)
Sets the commit policy.
|
void |
setConnection(Connection value)
Sets the actual JDBC connection associated with the scenario block.
|
void |
setConnectionName(String value)
Sets the name of the connection associated with the scenario block.
|
void |
setDataReaderClassName(String value)
Sets the data reader class name.
|
void |
setDataReaderParams(Map<String,String> value)
Sets the data reader params.
|
void |
setDataSet(DataSet value,
boolean adjust)
Sets the
DataSet object associated with the scenario block. |
void |
setDataWriterClassName(String value)
Sets the data writer class name.
|
void |
setDataWriterParams(Map<String,String> value)
Sets the data writer params.
|
void |
setDriverClassName(String value)
Sets the driver class name.
|
void |
setEnabled(boolean value)
Sets the enabled property of the scenario block.
|
void |
setEncoded(boolean value)
Sets the encoded flag for the
DataSet object associated with
the scenario block. |
void |
setExclude(String exclude)
Sets the list of the excluded tables for wildcard matching.
|
void |
setFetchSize(int fetchSize)
Sets the fetch size.
|
void |
setHighWatermarkField(String highWatermarkField)
Sets the high watermark field.
|
void |
setIgnoreEmpty(boolean ignoreEmpty)
Sets the ignore empty source flag.
|
void |
setIgnoreNoRecords(boolean ignoreNoRecords)
Sets the ignore no records in source flag.
|
void |
setInclude(String include)
Sets the list of the included tables for wildcard matching.
|
void |
setInlineTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed for each row of the
DataSet object associated with the scenario block. |
void |
setIsEmpty(boolean value)
Sets the state of the field
isEmpty for the scenario block
regardless of the actual state of the DataSet object. |
void |
setIsIndependent(Boolean value)
Sets the "is independent" flag.
|
void |
setKeyFields(String value)
Sets the name of the key fields.
|
void |
setKeyName(String value)
Sets the key name.
|
void |
setLinkedSourceName(String value)
Sets the name of the linked source.
|
void |
setLocation(boolean doSet,
String key,
org.w3c.dom.Node treesNode)
Set the location for the artifacts such as tasks, variables, etc.
|
void |
setLocation(boolean doSet,
String key,
TypedKeyValue<Point,Point> location)
Set the location for the artifacts such as tasks, variables, etc.
|
void |
setMandatory(boolean value)
Sets the "mandatory" flag.
|
void |
setMatch(String match)
Sets the match.
|
void |
setMaxFiles(int maxFiles)
Sets the max files to process when wildcard processing is enabled.
|
void |
setMaxRows(int value)
Sets the maximum number of rows to extract.
|
void |
setName(String value)
Sets the name.
|
void |
setNewConnection(boolean newConnection)
Sets the "new connection" flag.
|
void |
setNewConnectionSourceQuery(boolean newConnectionSourceQuery)
Sets the "new connection source query" flag.
|
void |
setNoConnection(boolean value)
Sets the "no connection" flag.
|
void |
setObjectName(String value)
Sets the name of the object.
|
void |
setOnExceptionTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed when there is an exception during processing of the etl block.
|
void |
setOnPersistDataSet(int value)
Sets the action for "on persist data set" event.
|
void |
setOnPersistDataSet(String action)
Sets the action for "on persist data set" event.
|
void |
setOnPopulateDataSet(int value)
Sets the action for "on populate data set" event.
|
void |
setOnPopulateDataSet(String action)
Sets the action for "on populate data set" event.
|
void |
setOriginalName(String name)
Sets the original block name.
|
void |
setOutputParamIndexes(String value)
Sets the comma delimited list of of indexes for the output parameters to
get a result set from callable statement.
|
void |
setOwnerName(String value)
Sets the owner name.
|
void |
setParallel(boolean value)
Sets the the property isParallel for the scenario block.
|
void |
setPostTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed after etl framework finished with the
processing of the particular scenario block.
|
void |
setSql(String value)
Sets the sql.
|
void |
setStatementType(SqlStatementType value)
Sets the statement type.
|
void |
setTasks(ListHashMap<String,Task> value)
Sets the tasks which are executed before etl framework starts processing
of the particular scenario block.
|
void |
setTrackingEnabled(boolean trackingEnabled)
Sets the tracking enabled.
|
void |
setUpdateIndexOnChange(boolean value)
Sets the value for the "update index on change event" flag.
|
void |
setUsing(String value)
Sets the "using".
|
void |
setVariables(ListHashMap<String,Variable> value)
Sets the variables for the block.
|
void |
setWildcardParallel(boolean wildcardParallel)
Sets the value of the flag which controls if parallel processing of wildcard sources is enabled.
|
findException, getExceptionActions, getExceptionMask, getMessage, getOnExceptionAction, getOnExceptionActions, handleException, hasAllExceptions, isSavePoint, log, setExceptionMask, setMessage, setOnExceptionAction, setOnExceptionAction, setSavePoint
getConditionCode, getConditionConnectionName, getConditionLang, setConditionCode, setConditionConnectionName, setConditionLang
public static final String SOURCE_ATTR
public static final String SOURCE_SQL_ATTR
public static final String NAME_ATTR
public static final String KEY_FIELDS_ATTR
public static final String SQL_STATEMENT_TYPE_ATTR
public static final String SQL_STATEMENT_PARAM_INDEX_ATTR
public static final String SQL_STATEMENT_OUTPUT_PARAM_INDEEX_ATTR
public static final String LIMIT_ATTR
public static final String MAX_FILES_ATTR
public static final String UPDATE_KEY_ATTR
public static final String OWNER_ATTR
public static final String OBJECT_NAME_ATTR
public static final String LINKED_SOURCE_ATTR
public static final String ON_PERSIST_ATTR
public static final String ON_POPULATE_ATTR
public static final String NO_CONNECTION_ATTR
public static final String PARALLEL_ATTR
public static final String WILDCARD_PARALLEL_ATTR
public static final String NEW_CONNECTION_ATTR
public static final String NEW_CONNECTION_SOURCE_QUERY_ATTR
public static final String INDEPENDENT_ATTR
public static final String EMPTY_ATTR
public static final String IGNORE_EMPTY_ATTR
public static final String IGNORE_NO_RECORDS_ATTR
public static final String MANDATORY_ATTR
public static final String ENABLED_ATTR
public static final String METRICS_ENABLED_ATTR
public static final String TRACKING_ENABLED_ATTR
public static final String ENCODE_ATTR
public static final String ETL_DRIVER_ATTR
public static final String CONNECTION_ATTR
public static final String SQL_ATTR
public static final String BEFORE_SQL_ATTR
public static final String USING_ATTR
public static final String EXCLUDE_ATTR
public static final String INCLUDE_ATTR
public static final String MATCH_ATTR
public static final String ON_EXCEPTION_ACTION_ATTR
public static final String ON_EXCEPTION_MASK_ATTR
public static final String COND_lANG_ATTR
public static final String COND_CONNECTION_ATTR
public static final String CONDITION_ATTR
public static final String READER_ATTR
public static final String WRITER_ATTR
public static final String FETCH_SIZE_ATTR
public void decUsageCounter()
public void enableMetrics(boolean value)
Block
enableMetrics
in interface Block
value
- the valuepublic ListHashMap<String,Task> getBeforeEtlTasks()
Block
DataSet
object associated with the particular scenario
block but after initializing is done. For example: after metadata are
collected.getBeforeEtlTasks
in interface Block
ListHashMap
objectpublic Block.BlockType getBlockType()
Block
getBlockType
in interface Block
public Connection getConnection()
Block
getConnection
in interface Block
public String getConnectionName()
Block
Source
or Destination
Example of named connection in the scenario file:
<destination>
<name>image</name>
<source>image</source>
<load>
<connection>mssql</connection>
<driver>com.toolsverse.etl.driver.mysql.MsSqlDriver</driver>
</load>
</destination>
getConnectionName
in interface Block
public String getDataReaderClassName()
public Map<String,String> getDataReaderParams()
public DataSet getDataSet()
Block
DataSet
object associated with the scenario block.getDataSet
in interface Block
public String getDataWriterClassName()
public Map<String,String> getDataWriterParams()
public String getDefaultConnectionName()
ConditionalExecution
getDefaultConnectionName
in class ConditionalExecution
public Map<String,Object> getDefaultValues(Scenario scenario)
scenario
- the scenariopublic String getDriverClassName()
Block
getDriverClassName
in interface Block
public Point getEndLocation()
LocationProvider
getEndLocation
in interface LocationProvider
public String getHighWatermarkField()
Block
getHighWatermarkField
in interface Block
public ListHashMap<String,Task> getInlineTasks()
Block
DataSet
object associated with the scenario block.getInlineTasks
in interface Block
ListHashMap
objectpublic String getKeyFields()
OnException
getKeyFields
in class OnException
public String getKeyName()
public String getLinkedSourceName()
public TypedKeyValue<Point,Point> getLocation(String key)
Block
getLocation
in interface Block
key
- the artifact type namepublic Map<String,TypedKeyValue<Point,Point>> getLocations()
Block
getLocations
in interface Block
public int getMaxRows()
public String getName()
OnException
getName
in interface Block
getName
in class OnException
public String getObjectName()
public ListHashMap<String,Task> getOnExceptionTasks()
Block
getOnExceptionTasks
in interface Block
ListHashMap
objectpublic int getOnPersistDataSet()
public int getOnPersistDataSet(String action)
action
- the actionpublic String getOnPersistDataSetAction()
public int getOnPopulateDataSet()
public int getOnPopulateDataSet(String action)
action
- the actionpublic String getOnPopulateDataSetAction()
public String getOutputParamIndexes()
public String getOwnerName()
public List<Integer> getParamIndexes()
public ListHashMap<String,Task> getPostTasks()
Block
getPostTasks
in interface Block
ListHashMap
objectpublic String getSql()
public Point getStartLocation()
LocationProvider
getStartLocation
in interface LocationProvider
public SqlStatementType getStatementType()
public ListHashMap<String,Task> getTasks()
Block
public int getUsageCounter()
public String getUsing()
public Variable getVariable(String... name)
Block
getVariable
in interface Block
name
- the name of the variablepublic ListHashMap<String,Variable> getVariables()
Block
getVariables
in interface Block
public void addVariable(Variable var)
var
- the variablepublic void incUsageCounter()
public boolean isEmpty()
Block
DataSet
object associated with the scenario block
is empty. Scenario block can be configured to be treated as empty
regardless of the actual state of the DataSet
object.public boolean isEnabled()
Block
public boolean isEncoded()
Block
DataSet
object associated
with the scenario block. Usually only Clob and Blob fields can be encoded. ETL
framework uses open source implementation of the Base64 encoding.public Boolean isIndependent()
public boolean isMandatory()
public boolean isMetricsEnabled()
Block
isMetricsEnabled
in interface Block
public boolean isParallel()
Block
isParallel
in interface Block
public boolean isStub()
public boolean isUpdateIndexOnChange()
public boolean noConnection()
Block
noConnection
in interface Block
public void setBeforeEtlTasks(ListHashMap<String,Task> value)
Block
DataSet
object associated with the particular scenario
block but after initializing is done. For example after metadata are
collected. Tasks get instantiated by parsing scenario file.setBeforeEtlTasks
in interface Block
value
- the new before etl taskspublic void setBlockType(Block.BlockType value)
value
- the new block typepublic void setConnection(Connection value)
Block
setConnection
in interface Block
value
- the new connectionpublic void setConnectionName(String value)
Block
setConnectionName
in interface Block
value
- the new name of the connectionpublic void setDataReaderClassName(String value)
value
- the new data reader class namepublic void setDataReaderParams(Map<String,String> value)
value
- the valuepublic void setDataSet(DataSet value, boolean adjust)
Block
DataSet
object associated with the scenario block.setDataSet
in interface Block
value
- the new data setadjust
- if true adjust data set fieldspublic void setDataWriterClassName(String value)
value
- the new data writer class namepublic void setDataWriterParams(Map<String,String> value)
value
- the valuepublic void setDriverClassName(String value)
Block
setDriverClassName
in interface Block
value
- the new driver class namepublic void setEnabled(boolean value)
Block
true
.
Example of the manually set enabled flag in the scenario file:
<source enabled="No">
<name>test</name>
<extract>
<sql>select * from test</sql>
</extract>
</source>
setEnabled
in interface Block
value
- the new enabledpublic void setEncoded(boolean value)
Block
DataSet
object associated with
the scenario block. By default all DataSet
objects which
contain Clob or Blob fields are encoded and compressed using zip
algorithm.
Example of the manually set encoded flag in the scenario file:
<source encode="No">
<name>clobs</name>
<extract>
<sql>select * from clobs</sql>
</extract>
</source>
setEncoded
in interface Block
value
- the new encodedpublic void setHighWatermarkField(String highWatermarkField)
highWatermarkField
- the new high watermark fieldpublic void setInlineTasks(ListHashMap<String,Task> value)
Block
DataSet
object associated with the scenario block. Inline
tasks get instantiated by parsing scenario file.
Example of the inline task in the scenario file:
<tasks>
<task>
<name>replace</name>
setInlineTasks
in interface Block
value
- the value
<class>com.toolsverse.etl.core.task.common.RegexpTransformator</class>
<variables>
<FIELD value="DESCRIPTION" />
<REGEXP value="Substitution" />
<REPLACE value="something" />
</variables>
</task>
</tasks>
public void setIsEmpty(boolean value)
Block
isEmpty
for the scenario block
regardless of the actual state of the DataSet
object. In the
majority of cases etl framework can automatically set this flag however
it doesn't hurt to do it manually.
Example of the always empty scenario block in the scenario file:
<source empty="Yes">
<name>init</name>
<extract>
<tasks>
<task>
<name>prepare</name>
<class>com.toolsverse.etl.core.task.sql.SqlTask</class>
<sql>drop table tmp_cp</sql>
<onexception action="ignore"/>
</task>
</tasks>
</extract>
</source>
setIsEmpty
in interface Block
value
- the new isEmptypublic void setIsIndependent(Boolean value)
value
- the new value for "is independent" flagpublic void setKeyFields(String value)
OnException
setKeyFields
in class OnException
value
- the new names of the key fieldspublic void setKeyName(String value)
value
- the new key namepublic void setLinkedSourceName(String value)
value
- the new name of the linked source.public void setLocation(boolean doSet, String key, org.w3c.dom.Node treesNode)
Block
setLocation
in interface Block
doSet
- if false do not add location info for the artifactkey
- the artifact typetreesNode
- the DOM node which holds location informationpublic void setLocation(boolean doSet, String key, TypedKeyValue<Point,Point> location)
Block
setLocation
in interface Block
doSet
- if false do not add location info for the artifactkey
- the artifact typelocation
- the locationpublic void setMandatory(boolean value)
value
- the new value for the "mandatory" flagpublic void setMaxRows(int value)
value
- the new maximum number of rows to extractpublic void setName(String value)
OnException
setName
in interface Block
setName
in class OnException
value
- the new namepublic void setNoConnection(boolean value)
value
- the new value for the "no connection" flagpublic void setObjectName(String value)
Example of the objectname
at the source level in the
scenario file:
<source>
<name>test</name>
<objectname>ABC.CLIENT</objectname>
</source>
value
- The new name of the objectpublic void setOnExceptionTasks(ListHashMap<String,Task> value)
Block
Example of the post task in the scenario file:
<tasks>
<task scope="onexception">
<name>delete</name>
<class>com.toolsverse.etl.core.task.common.FileManagerTask</class>
<variables>
<COMMAND value="delete" />
<SOURCE_TYPE value="file" />
<FILES value="*.dat" />
</variables>
</task>
</tasks>
setOnExceptionTasks
in interface Block
value
- the new post taskspublic void setOnPersistDataSet(int value)
value
- the new action for the "on persist data set" eventpublic void setOnPersistDataSet(String action)
action
- the new action for "on persist data set" eventpublic void setOnPopulateDataSet(int value)
value
- the new action for the "on populate data set" eventpublic void setOnPopulateDataSet(String action)
action
- the new action for "on populate data set" eventpublic void setOutputParamIndexes(String value)
value
- the new comma delimited list of of indexespublic void setOwnerName(String value)
value
- the new owner namepublic void setParallel(boolean value)
Block
true
the code for this block can be executed in the separate
thread. The default value is false
.
Example of the parallel scenario block in the scenario file:
<source parallel="Yes">
<name>test</name>
<extract>
<sql>select * from test</sql>
</extract>
</source>
setParallel
in interface Block
value
- the new parallelpublic void setPostTasks(ListHashMap<String,Task> value)
Block
Example of the post task in the scenario file:
<tasks>
<task scope="after">
<name>delete</name>
<class>com.toolsverse.etl.core.task.common.FileManagerTask</class>
<variables>
<COMMAND value="delete" />
<SOURCE_TYPE value="file" />
<FILES value="*.dat" />
</variables>
</task>
</tasks>
setPostTasks
in interface Block
value
- the new post taskspublic void setSql(String value)
value
- the new sqlpublic void setStatementType(SqlStatementType value)
value
- the new statement typepublic void setTasks(ListHashMap<String,Task> value)
Block
Example of the task in the scenario file:
<tasks>
<task>
<name>copy</name>
<connection>dest</connection>
<class>com.toolsverse.etl.core.task.oracle.OracleCopyTask</class>
<sql>
insert temp USING
select * from some_table
</sql>
</task>
</tasks>
public void setUpdateIndexOnChange(boolean value)
value
- the new value for the "update index on change event" flagpublic void setUsing(String value)
value
- the new "using"public void setVariables(ListHashMap<String,Variable> value)
Block
Example of the variables in the scenario file:
<variables>
<STEP_NUM function="getPk" />
<CONFIG_NUM function="getFk" />
<USER function="getUser" />
<ID include="No" />
</variables>
setVariables
in interface Block
value
- The new variablespublic boolean isIgnoreEmpty()
public void setIgnoreEmpty(boolean ignoreEmpty)
ignoreEmpty
- the value of the ignore empty source flagpublic boolean isIgnoreNoRecords()
public void setIgnoreNoRecords(boolean ignoreNoRecords)
ignoreNoRecords
- the value of the ignore no records in the source flagpublic Map<String,String> getFiles()
ReadTracker
getFiles
in interface ReadTracker
public boolean isTrackingEnabled()
ReadTracker
isTrackingEnabled
in interface ReadTracker
public void setTrackingEnabled(boolean trackingEnabled)
trackingEnabled
- the new tracking enabledpublic String getOriginalName()
Block
getOriginalName
in interface Block
public void setOriginalName(String name)
Block
setOriginalName
in interface Block
name
- String the new original namepublic int getFetchSize()
public void setFetchSize(int fetchSize)
fetchSize
- the new fetch sizepublic String getIdForException()
ConditionalExecution
getIdForException
in class ConditionalExecution
public String getExclude()
public void setExclude(String exclude)
exclude
- the new excludepublic String getInclude()
public void setInclude(String include)
include
- the coma separated list of included tables for the wildcard
matchingpublic void setCommitPolicy(StreamingCommitPolicy commitPolicy)
commitPolicy
- the new commit policypublic StreamingCommitPolicy getCommitPolicy()
public boolean isWildcardParallel()
public void setWildcardParallel(boolean wildcardParallel)
wildcardParallel
- booleanpublic String getMatch()
public void setMatch(String match)
match
- the new matchpublic boolean isNewConnection()
public void setNewConnection(boolean newConnection)
newConnection
- booleanpublic int getMaxFiles()
public void setMaxFiles(int maxFiles)
maxFiles
- the new max files to processpublic String getBeforePrepareSourceQuery()
public void setBeforePrepareSourceQuery(String beforePrepareSourceQuery)
beforePrepareSourceQuery
- Stringpublic boolean isNewConnectionSourceQuery()
public void setNewConnectionSourceQuery(boolean newConnectionSourceQuery)
newConnectionSourceQuery
- booleanCopyright © 2010-2020 Toolsverse. All Rights Reserved.