public class TextConnector extends BaseDataSetConnector implements DataSetConnector<TextConnectorParams,ConnectorResult>, StreamingConnector<TextStreamingPackage,TextProducerPackage>
DEFAULT_IMPORT
APP, COMMAND, ETL_DRIVER, FORMATTER, IDE_NODE, NO_RANK, WIDGET
Constructor and Description |
---|
TextConnector() |
Modifier and Type | Method and Description |
---|---|
void |
beforeStream(TextProducerPackage producerPackage)
Executed before record has been streamed.
|
boolean |
canRead()
Checks if connector can read data.
|
boolean |
canWrite()
Checks if connector can write data.
|
void |
cleanUp(TextConnectorParams params,
DataSet dataSet,
Driver driver)
Cleans up after DataSetConnector populated or persisted DataSet.
|
void |
closeStream(TextStreamingPackage streamingPackage)
Close stream.
|
boolean |
consume(TextStreamingPackage streamingPackage,
Object message,
AtomicInteger row)
Consume the next record.
|
void |
endStream(TextProducerPackage producerPackage)
Executed after record has been streamed.
|
TextConnectorParams |
getDataSetConnectorParams()
Gets the DataSetConnectorParams.
|
String |
getDisplayName()
Gets the display name.
|
static int[] |
getLengthArray(TextConnectorParams params,
int fieldCount)
Gets the array of lengths for each field in the fixed-length file.
|
String |
getName()
Gets the name.
|
String |
getStringToPersist(String value,
boolean stripNotPrintable,
boolean removeEol)
Gets the string to persist.
|
String |
getVendor()
Gets the vendor.
|
String |
getVersion()
Gets the version of the extension module.
|
String |
getXmlConfigFileName()
Gets the path to the xml configuration file for the module.
|
String |
initQueryLabel()
The init query label.
|
void |
inlinePersist(TextConnectorParams params,
DataSet dataSet,
Driver driver,
DataSetRecord record,
int row,
int records)
Persists data set row.
|
boolean |
isFileConnector()
Checks if this is a connector to the file based data source.
|
boolean |
isTextFormat()
Checks if this is a connector for the text-based data format, such as
JOSN, XML, etc.
|
String[] |
parseLineWithCharSeparators(String line,
String separator,
String charSeparator,
Character quotePrefix,
org.apache.commons.csv.CSVFormat format)
Parses the line with char separators.
|
ConnectorResult |
persist(TextConnectorParams params,
DataSet dataSet,
Driver driver)
Persists data set.
|
ConnectorResult |
populate(TextConnectorParams params,
DataSet dataSet,
Driver driver)
Populates data set.
|
void |
postPersist(TextConnectorParams params,
DataSet dataSet,
Driver driver)
Called when DataSetConnector finished persisting data set.
|
void |
prePersist(TextConnectorParams params,
DataSet dataSet,
Driver driver)
Called when DataSetConnector starts persisting data set.
|
void |
startStream(TextProducerPackage producerPackage)
Start the stream.
|
boolean |
supportsNested()
Supports nested.
|
boolean |
supportsReadStreaming()
Supports read streaming.
|
boolean |
supportsReordering(TextConnectorParams params)
Supports reordering of the fields.
|
boolean |
supportsWriteStreaming()
Supports write streaming.
|
ConnectorResult |
writeMetaData(TextConnectorParams params,
DataSet dataSet,
Driver driver)
Writes meta data.
|
addValue, basePostPersist, getCommitPolicy, getExt, getInputStream, getIoProcessor, getLicensePropertyName, getReader, getType, postPersistWhenEmpty, setCommitPolicy, stream, supportsWorksheets, testConnection
compareTo, getConfigFileName, getExtensionRank, getIconPath, getLocalUnitClassPath
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getExt, supportsWorksheets, testConnection
getCommitPolicy, setCommitPolicy
public boolean canRead()
DataSetConnector
canRead
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public boolean canWrite()
DataSetConnector
canWrite
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public void cleanUp(TextConnectorParams params, DataSet dataSet, Driver driver) throws Exception
DataSetConnector
cleanUp
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the DataSetConnectorParamsdataSet
- the data setdriver
- the driverException
- in case of any errorpublic TextConnectorParams getDataSetConnectorParams()
DataSetConnector
getDataSetConnectorParams
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public String getDisplayName()
ExtensionModule
getDisplayName
in interface ExtensionModule
public static int[] getLengthArray(TextConnectorParams params, int fieldCount)
params
- the TextConnectorParamsfieldCount
- the field countpublic String getName()
DataSetConnector
getName
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public String getVendor()
ExtensionModule
getVendor
in interface ExtensionModule
public String getVersion()
ExtensionModule
getVersion
in interface ExtensionModule
public String getXmlConfigFileName()
ExtensionModule
getXmlConfigFileName
in interface ExtensionModule
getXmlConfigFileName
in class BaseExtension
public void inlinePersist(TextConnectorParams params, DataSet dataSet, Driver driver, DataSetRecord record, int row, int records) throws Exception
DataSetConnector
inlinePersist
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the DataSetConnectorParamsdataSet
- the data setdriver
- the driverrecord
- the recordrow
- the rowrecords
- the total number of recordsException
- in case of any errorpublic String getStringToPersist(String value, boolean stripNotPrintable, boolean removeEol)
value
- the valuepublic boolean isFileConnector()
DataSetConnector
isFileConnector
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public boolean isTextFormat()
DataSetConnector
isTextFormat
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public String[] parseLineWithCharSeparators(String line, String separator, String charSeparator, Character quotePrefix, org.apache.commons.csv.CSVFormat format) throws Exception
line
- the lineseparator
- the separatorcharSeparator
- the char separatorException
- the exceptionpublic ConnectorResult persist(TextConnectorParams params, DataSet dataSet, Driver driver) throws Exception
DataSetConnector
persist
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the DataSetConnectorParamsdataSet
- the data setdriver
- the driverException
- in case of any errorpublic boolean consume(TextStreamingPackage streamingPackage, Object message, AtomicInteger row) throws Exception
StreamingConnector
consume
in interface StreamingConnector<TextStreamingPackage,TextProducerPackage>
streamingPackage
- the streaming packagemessage
- the message to consumerow
- the current row numberException
- in case of any errorpublic void beforeStream(TextProducerPackage producerPackage) throws Exception
StreamingConnector
beforeStream
in interface StreamingConnector<TextStreamingPackage,TextProducerPackage>
producerPackage
- the producer packageException
- in case of any errorpublic void startStream(TextProducerPackage producerPackage) throws Exception
StreamingConnector
startStream
in interface StreamingConnector<TextStreamingPackage,TextProducerPackage>
producerPackage
- the producer packageException
- the exceptionpublic void endStream(TextProducerPackage producerPackage) throws Exception
StreamingConnector
endStream
in interface StreamingConnector<TextStreamingPackage,TextProducerPackage>
producerPackage
- the producer packageException
- in case of any errorpublic void closeStream(TextStreamingPackage streamingPackage)
StreamingConnector
closeStream
in interface StreamingConnector<TextStreamingPackage,TextProducerPackage>
streamingPackage
- the streaming packagepublic ConnectorResult populate(TextConnectorParams params, DataSet dataSet, Driver driver) throws Exception
DataSetConnector
populate
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the DataSetConnectorParamsdataSet
- the data setdriver
- the driverException
- in case of any errorpublic void postPersist(TextConnectorParams params, DataSet dataSet, Driver driver) throws Exception
DataSetConnector
postPersist
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the DataSetConnectorParamsdataSet
- the data setdriver
- the driverException
- in case of any errorpublic void prePersist(TextConnectorParams params, DataSet dataSet, Driver driver) throws Exception
DataSetConnector
prePersist
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the DataSetConnectorParamsdataSet
- the data setdriver
- the driverException
- in case of any errorpublic ConnectorResult writeMetaData(TextConnectorParams params, DataSet dataSet, Driver driver) throws Exception
DataSetConnector
writeMetaData
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the DataSetConnectorParamsdataSet
- the data setdriver
- the driverException
- in case of any errorpublic String initQueryLabel()
DataSetConnector
initQueryLabel
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public boolean supportsReadStreaming()
DataSetConnector
supportsReadStreaming
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public boolean supportsWriteStreaming()
DataSetConnector
supportsWriteStreaming
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public boolean supportsNested()
DataSetConnector
supportsNested
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
public boolean supportsReordering(TextConnectorParams params)
DataSetConnector
supportsReordering
in interface DataSetConnector<TextConnectorParams,ConnectorResult>
params
- the paramsCopyright © 2010-2020 Toolsverse. All Rights Reserved.