avro file connector

Description - Emits a document to an Avro container file

Runtime Class - com.ibi.agents.XDAvroFileEmitAgent



Parameter Groups

 ParameterTypeDescription
*Directory:stringThe target output directory
*File Pattern:stringThe output file name, which can contain a '*' which gets expanded to a fine timestamp
Avro Schema:stringPath to the Avro Schema file. If absent, the schema stored with the document will be used.
Codec:stringAlgorithm used to compress the data.

Select from one of the following options:
  • BZip2 {bzip2}
  • Deflate {deflate}
  • No Compression {null}
  • Snappy {snappy}
  • XZ {xz}
* - denotes required
 ParameterTypeDescription
*Transactional:booleanIf set, the write operation respects the transactionality of the channel. If not set, the emit is always done.
* - denotes required
 ParameterTypeDescription
Output Document:stringSets the outbound document of this object.

Select from one of the following options:
  • Status document {status} - Returns a status document.
  • Input document {input} - Returns the input document of the object.
* - denotes required

Output Events

NameDescription
OnParseErrorCould not parse a document.
OnFailedOperationCould not perform the operation requested.
Description - Read data in Avro format

Runtime Class - com.ibi.agents.XDAvroFileReadAgent



Parameter Groups

 ParameterTypeDescription
Schema:stringPath to the Avro Schema file. If absent, the schema stored with the data will be used.
Format:stringFormat of the incoming Avro data after conversion

Select from one of the following options:
  • XML {xml}
  • JSON {json}
* - denotes required

Output Events

NameDescription
OnParseErrorCould not parse a document.
OnNotFoundThe resource was not found and this is considered an error.
OnFailedOperationCould not perform the operation requested.
Description - Read data in Avro format

Runtime Class - com.ibi.agents.XDAvroFileReadAgent


Configuration

This action requires a Generics Configuration as well as the following required parameters.

Parameter Groups

 ParameterTypeDescription
Schema:stringPath to the Avro Schema file. If absent, the schema stored with the data will be used.
*Data File:stringPath to the Avro data file.
Format:stringFormat of the incoming Avro data after conversion

Select from one of the following options:
  • XML {xml}
  • JSON {json}
* - denotes required

Output Events

NameDescription
OnParseErrorCould not parse a document.
OnNotFoundThe resource was not found and this is considered an error.
OnFailedOperationCould not perform the operation requested.
Description - Send an Avro RPC request over HTTP

Runtime Class - com.ibi.agents.XDAvroRpcHttpEmitAgent


Configuration

This action requires a Generics Configuration as well as the following required parameters.

Parameter Groups

 ParameterTypeDescription
*File Path:stringPath to the Avro Protocol file. Typical extension is .avpr
*Message Name:stringSelects the message within the Avro Protocol
* - denotes required
 ParameterTypeDescription
Namespace:stringThe special register namespace from which HTTP headers for the outgoing request will be taken.

Select from one of the following options:
  • Default Namespace {default}
  • None {none}
Excluded HeadersstringComma delimited list (case insensitive) of headers that should not be sent with request, even if found in request header namespace.
* - denotes required
 ParameterTypeDescription
Namespace:stringThe special register namespace into which HTTP headers from the incoming response will be saved.

Select from one of the following options:
  • Default Namespace {default}
* - denotes required
 ParameterTypeDescription
*Return:stringReturn from this object.

Select from one of the following options:
  • Response Document {response} - Returns the results of the operation.
  • Status Document {status} - Returns a status document
  • Input Document {input} - Return the document passed in
* - denotes required

Output Events

NameDescription
OnFailConnectionCould not connect to the external system.
OnFailRedirectionRedirection status code between 300 and 399.
OnFailClientErrorClient error status code between 400 and 499.
OnFailServerErrorServer error status code between 500 and 599.
OnFailedOperationCould not perform the operation requested.
OnParseErrorCould not parse a document.
OnFailLargeRequestRequest entity size exceeds configured maximum.
OnFailLargeResponseResponse entity size exceeds configured maximum.
Subscribe to avro file connector