Hi All,
This post will be an introduction to the Global Parameters available in Azure Data Factory V2.
Using PowerShell to Setup Performance Monitor Data Collector Sets. Updated 2020-04-02 for 0x80300103 fix.
Setting up Performance Monitor Data Collector Sets can be a time consuming exercise, especially when you have to do it on multiple servers.
Using sources from multiple places, here is some code I use to create a Data Collector Set, and inside that a number of Data Collectors for Microsoft SQL Server instances running on the server.
Azure Data Factory V2 – Variables
Hi All,
I saw these in the General Activity lists the other day, so I quickly jumped onto the Microsoft documentation, and found there wasn’t much about how to use these.
So, I’ll outline a short example, along with a link to the ARM template to deploy into your own Data Factory.
Azure Data Factory V2 – Filter Activity
Hi All. Staying with the Data Factory V2 theme for this blog. Version 2 introduced a few Iteration & Conditionals activities.
One of these is the Filter activity.
There is a number of use cases for this activity, such as filtering the outputs from the Get Metadata and Lookup Activities.
In this example, I’m going to show you a few ways to apply filtering using the Filter with a few Pipeline parameters. The last example, which I’ll also include the ARM template for, is dynamic for both the Operator, and also the item being filtered.
Continue Reading…
Azure Data Factory V2 – Handling Daylight Savings using Azure Functions – Page 2.
In my previous blog, we set up an Azure Function to return current daylight saving time status for a timezone.
In this blog, we will create a new Version 2 Data Factory, and import an ARM template with a Pipeline to adjust for daylight savings. We can import this template into an existing Data Factory if required.
Continue Reading…
Azure Data Factory V2 – Handling Daylight Savings using Azure Functions – Page 1.
Hi Everyone.
Apologies for the lack of posts. I will try to post more often!!
Since my last blog where I was meant to follow up and go through Data Factory Custom Activities, Azure Data Factory Version 2 has been released into Preview.
As of today, I still don’t believe the ‘Triggers’ for scheduling a Pipeline allow us to set a timezone, so we have to schedule at UTC.
In this blog, I will go through one of the ways we can work around this whilst creating the triggers in Data Factory itself. (It can also be done through Automation)
Continue Reading…
Azure Data Factory Custom Activities – Setting up Azure Batch Service
In this blog I will give brief instructions to setting up an Azure Batch Service, which is required to execute C# code through Custom Activities via Azure Data Factory.
Continue Reading…
Attaching Tasks to Event in Windows
In my previous blog, we found an error in the Event Log after the scheduled restart of the Windows Task Scheduler task for the Performance Monitor Data Collector Set.
Data collector set SQLAudit failed to start as WORKGROUP\xxxxx$ with error code 0x803000AA.
In this blog, I will outline how we can attach a Task to this event, which will execute a script to start the original task.
Configuring Windows Task Scheduler to run daily Performance Monitor Data Collector Set.
In my previous blog, I posted a PowerShell script that can configure a Data Collector Set, with Data Collectors for the server and all MS SQL instances.
As part of my testing, I found that setting up Windows Task Scheduler to restart daily at 5am, that the Perfmon data collector set was not restarting.