Azure Data Factory V2 – Handling Daylight Savings using Azure Functions – Page 2.

In my previous blog, we set up an Azure Function to return current daylight saving time status for a timezone.

In this blog, we will create a new Version 2 Data Factory, and import an ARM template with a Pipeline to adjust for daylight savings. We can import this template into an existing Data Factory if required.

To begin, sign into the Azure Portal using an account with permissions to create new resources / services.

Click on the + Create a resource to create a new resource..


 
 
 

Type Data Factory into the search field and press Enter.

Select Data Factory , publisher Microsoft.

Click Create

Fill in details for the new services

  • Type in a Data Factory name.
  • Select the Subscription.
  • Select either Create new or Use existing resource group.
  • Select V2 (Preview)
  • Select a Location.

Click Create, and in a few minutes the Data Factory will be created.

Now we can import the ARM template with the Pipeline.
Go to Azure Custom Deployment to open the custom deployment page in Azure. Sign in if you need to.

Select Build your own template in the editor

Replace the text in the template with the following JSON.

{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "factoryName": {
            "type": "string",
            "metadata": "Data Factory Name",
            "defaultValue": "Azure Data Factory"
        },
        "dstCheckURL": {
            "type": "string",
            "metadata": "URL for the DST Check function"
        }
    },
    "variables": {
        "factoryId": "[concat('Microsoft.DataFactory/factories/', parameters('factoryName'))]"
    },
    "resources": [
        {
            "name": "[concat(parameters('factoryName'), '/PL DST Adjustment')]",
            "type": "Microsoft.DataFactory/factories/pipelines",
            "apiVersion": "2017-09-01-preview",
            "properties": {
                "description": "Pipeline to adjust trigger run time depending on DST status.",
                "activities": [
                    {
                        "name": "Get DST Flag",
                        "description": "Call Azure Function to check current DST status.",
                        "type": "WebActivity",
                        "dependsOn": [],
                        "policy": {
                            "timeout": "7.00:00:00",
                            "retry": 0,
                            "retryIntervalInSeconds": 30,
                            "secureOutput": false
                        },
                        "userProperties": [],
                        "typeProperties": {
                            "url": "[parameters('dstCheckURL')]",
                            "method": "GET",
                            "headers": {}
                        }
                    },
                    {
                        "name": "Check DST Flag",
                        "description": "Check DST Flag and Delay 3600 Seconds (1 Hour) when FALSE",
                        "type": "IfCondition",
                        "dependsOn": [
                            {
                                "activity": "Get DST Flag",
                                "dependencyConditions": [
                                    "Succeeded"
                                ]
                            }
                        ],
                        "userProperties": [],
                        "typeProperties": {
                            "expression": {
                                "value": "@activity('Get DST Flag').isDST",
                                "type": "Expression"
                            },
                            "ifFalseActivities": [
                                {
                                    "name": "DST False",
                                    "description": "DST False - Delay 3600 Seconds.",
                                    "type": "Wait",
                                    "dependsOn": [],
                                    "userProperties": [],
                                    "typeProperties": {
                                        "waitTimeInSeconds": 3600
                                    }
                                }
                            ]
                        }
                    }
                ]
            },
            "dependsOn": []
        }
    ]
}

And press Save

The Template page should be displayed. Fill out the details as required, and click Purchase

This should deploy the pipeline into your Data Factory.
When that is complete. Open your Data Factory, select Author and Monitor
From the Let’s get started screen, select the pencil/Author icon on the left.

Click on Pipeline to expand. Hopefully a PL DST Adjustment pipeline is present.

To use this Pipeline, create a General / Execute Pipeline activity in your Pipeline, and in the Settings tab use the PL DST Adjustment pipeline as the Invoked Pipeline

Don’t forget to click on Advanced and set the Wait on completion setting so the current pipeline waits for the execution to complete.
This will give your pipeline a 60 minute delay when daylight savings is FALSE.

Note: When scheduling your pipeline, Schedule it to run at the time it will run when daylight savings is in place.

Don’t forget to publish after completing your changes. Please feel free to let me know if there are any issues with this, or you would like more details on anything.

Thank you for visiting!!

Leave a Reply

Your email address will not be published. Required fields are marked *