Monday, April 13, 2020

Create Delta Table from JSON File

Requirement:
                   
                     I will receive transaction files from vendor frequently. They provide table header file and data file separately. Our intention is create table based on the header file and load the data using data file.

Code Work Flow:

                     Here is the steps involved,

Step 1: Read the header json file and generate DDL


Step 2: Read the data file



Step 3: Create temporary view based on the data file created in step 2


Step 4: Create table DDL



Step 5: Create Table and Load the data



Step 6: Result



No comments:

Call Data bricks Job Using REST API

Below power shell will help to call Data bricks Job with parameter  [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]...