Package | Description |
---|---|
com.toolsverse.etl.core.codegen | |
com.toolsverse.etl.core.engine |
Modifier and Type | Method and Description |
---|---|
void |
DefaultCodeGen.addScriptToCreate(Destination destination,
String script) |
void |
DefaultCodeGen.addScriptToInit(Destination destination,
String script,
boolean cleanUpOnException,
String type,
boolean ignoreError) |
void |
DefaultCodeGen.execute(EtlConfig config,
Scenario scenario,
Driver driver,
boolean silent,
Connection conn,
Connection cleanUpConn,
Destination destination) |
void |
DefaultCodeGen.prepare(EtlConfig config,
Scenario scenario,
Destination destination,
boolean silent,
boolean onlyInit,
boolean isStreaming) |
void |
DefaultCodeGen.reorderColumns(Destination destination,
DataSetConnector writer,
DataSetConnectorParams writerParams) |
void |
DefaultCodeGen.runRemainingBatch(EtlConfig config,
Scenario scenario,
Connection conn,
Destination destination) |
Modifier and Type | Method and Description |
---|---|
Destination |
LoadFunctionContext.getDestination()
Gets the current
Destination . |
Destination |
Destination.getDestination()
Gets the linked destination.
|
Destination |
Scenario.getDestinationBySource(Source source)
Gets the destination by source.
|
Modifier and Type | Method and Description |
---|---|
ListHashMap<String,Destination> |
Scenario.getDestinations()
Gets the destinations.
|
Modifier and Type | Method and Description |
---|---|
void |
Scenario.addDestination(Destination destination)
Adds the destination.
|
void |
CodeGen.addScriptToCreate(Destination destination,
String script)
Adds the script to create.
|
void |
CodeGen.addScriptToInit(Destination destination,
String script,
boolean cleanUpOnException,
String type,
boolean ignoreError)
Adds the script to init.
|
void |
MetadataTracker.createTableAndIndexesOrChangeFieldDataTypeIfNeeded(Scenario scenario,
EtlConfig config,
Destination destination,
DataSet dataSet,
Alias alias,
CodeGen codeGen)
Creates the table and indexes or change field data type if needed.
|
void |
CodeGen.execute(EtlConfig config,
Scenario scenario,
Driver driver,
boolean silent,
Connection conn,
Connection cleanUpConn,
Destination destination)
Executes code.
|
void |
Extractor.extract(EtlConfig config,
Scenario scenario,
Source source,
Destination destination,
AddRecordCallback addRecordCallback)
Extracts data from the source.
|
void |
EtlMetricsTracker.finishTrackingLoad(String name,
Destination destination,
String exception)
Finish tracking load.
|
Object |
EtlMetricsTracker.getHighWatermark(EtlConfig config,
Scenario scenario,
Source source,
Destination destination,
Boolean... onLoad)
Gets the high watermark value.
|
String |
MetadataTracker.getLoadKey(EtlConfig config,
Destination destination,
String destTableName,
List<String> fields,
DataSetFields dataSetFields,
Driver driver)
Gets the load key.
|
String |
MetadataTracker.getSourceKeys(Destination destination,
EtlConfig config,
DataSetFields dataSetFields)
Gets the source keys.
|
boolean |
MetadataTracker.needToUpdateMetadataInTarget(Destination destination,
DataSet dataSet)
Checks if ETL engine needs to update metadata for the destination based
on the existing data.
|
void |
CodeGen.prepare(EtlConfig config,
Scenario scenario,
Destination destination,
boolean silent,
boolean onlyInit,
boolean isStreaming)
Prepares code for the destination using given config and scenario.
|
TypedKeyValue<String,String> |
EtlMetricsTracker.prepareTrackingExtract(Scenario scenario,
Source source,
Destination destination,
String sql,
EtlConfig config)
Prepare tracking extract.
|
void |
TaskRecorder.recordFinishHwmQuery(Destination destination)
Record event when the HWM query has finished running.
|
void |
TaskRecorder.recordFinishLoad(Destination destination)
Record event when the HWM query has finished running.
|
void |
TaskRecorder.recordStartHwmQuery(Destination destination,
String code)
Record event when the HWM query has started.
|
void |
TaskRecorder.recordStartLoad(Destination destination)
Record event when the load has started.
|
boolean |
MetadataTracker.recreateDestTableIfNeededWhenHWMIsEnabled(Scenario scenario,
Source source,
Destination destination,
EtlConfig config)
Recreate destination table if needed when the high watermark replication
is enabled.
|
void |
CodeGen.reorderColumns(Destination destination,
DataSetConnector writer,
DataSetConnectorParams writerParams)
Reorder columns.
|
void |
CodeGen.runRemainingBatch(EtlConfig config,
Scenario scenario,
Connection conn,
Destination destination)
Run remaining batch statements.
|
void |
LoadFunctionContext.setDestination(Destination value)
Sets the current
Destination . |
void |
Destination.setDestination(Destination value)
Sets the destination, linked to the destination.
|
void |
EtlMetricsTracker.startTrackingExtract(String name,
Source source,
DataSet dataSet,
Destination destination,
String flow)
Start tracking extract.
|
void |
EtlMetricsTracker.startTrackingLoad(String name,
Destination destination,
String flow)
Start tracking load.
|
void |
MetadataTracker.updateMetadataInTarget(EtlConfig config,
Destination destination,
DataSet dataSet,
Scenario scenario)
Update metadata in target.
|
void |
MetadataTracker.updateOrderOfFields(Destination destination,
DataSet dataSet)
Update order of fields.
|
Modifier and Type | Method and Description |
---|---|
void |
Scenario.setDestinations(ListHashMap<String,Destination> value)
Sets the destinations.
|
Copyright © 2010-2020 Toolsverse. All Rights Reserved.