After data is ingested to a Data Lakehouse, what happens if the data is structurally modified by a process outside of Infoworks and the metastore records are changed?

Sep 23, 2025 · The getting started chapter will walk you through the following Infoworks processes: now 6.2.1. Kubernetes-Based Deployment for AKS Configuring and Managing …

Infoworks agile data engineering platform automates the creation and operation of big data workflows from source to consumption, both on premise and in the cloud.

Navigate to Admin section of Infoworks UI, and click on external scripts, which shows up pipeline extensions page. Click on Add an extension,and configure it as follows:

This service listens to the changes on the source cluster HDFS, and writes the changes in the Infoworks metastore. The incremental replicator will then replicate the changes after reading it …

You can run any custom script with any required libraries in the Bash Node in the Workflow in Kubernetes Infoworks. The script will be accessible to this container in the form of a mounted …

For more details, refer to our Knowledge Base and Best Practices! For help, contact our support team! (C) 2015-2022 Infoworks.io, Inc. and Confidential

Infoworks will partition the column range into slices based on the number of connections allowed. Example: A table with a user_id column ranging from 1 to 1000, split into 10 ranges, will result …

To modify the Precision and Scale for a column in the table in the Infoworks source, ensure the source and target data are the same and there are no corrupt/modified values in the target …

Step 4: Configure a Custom Role on Infoworks If the default role is not sufficient, you can explicitly pass a different role in the platform configuration: Go to your Snowflake Data Environment …