![]() Topics are divided into a set of logs known as partitions. Kafka records are stored within topics, and consist of a category to which the Passed downstream to any other steps included within the same parent transformation. Processed by a Kafka Consumer step in a parent transformation to be You can set up this step to collect monitored events, track user consumptionĪdditionally, from the Kafka Consumer step, you can select a step in theĬhild transformation to stream records back to the parent transformation. To accept for processing, as well as the specific data formats to stream activity data and In the Kafka Consumer step itself, you can define the number of messages If you are using Spark as the processing engine, you must execute the child transformation according to Duration (ms) only. ![]() The Adaptive Execution Layer (AEL), using Spark as the processing engine. Depending on your setup, you can execute the ![]() ![]() You can configure the Kafka Consumer step to continuously ingest The child transformation must start with the Get records from stream step. That executes according to message batch size or duration, letting you process a continuous stream of records in near real-time. The parent Kafka Consumer step runs a child (sub-transformation) The PDI client pulls streaming data from Kafka through a Kafka transformation. ![]()
0 Comments
Leave a Reply. |