Azure Data Factory V2 – Filter Activity

Hi All. Staying with the Data Factory V2 theme for this blog. Version 2 introduced a few Iteration & Conditionals activities.
One of these is the Filter activity.
There is a number of use cases for this activity, such as filtering the outputs from the Get Metadata and Lookup Activities.
In this example, I’m going to show you a few ways to apply filtering using the Filter with a few Pipeline parameters. The last example, which I’ll also include the ARM template for, is dynamic for both the Operator, and also the item being filtered.
Continue Reading…

Azure Data Factory V2 – Handling Daylight Savings using Azure Functions – Page 2.

In my previous blog, we set up an Azure Function to return current daylight saving time status for a timezone.

In this blog, we will create a new Version 2 Data Factory, and import an ARM template with a Pipeline to adjust for daylight savings. We can import this template into an existing Data Factory if required.
Continue Reading…

Azure Data Factory V2 – Handling Daylight Savings using Azure Functions – Page 1.

Hi Everyone.
Apologies for the lack of posts. I will try to post more often!!

Since my last blog where I was meant to follow up and go through Data Factory Custom Activities, Azure Data Factory Version 2 has been released into Preview.
As of today, I still don’t believe the ‘Triggers’ for scheduling a Pipeline allow us to set a timezone, so we have to schedule at UTC.
In this blog, I will go through one of the ways we can work around this whilst creating the triggers in Data Factory itself. (It can also be done through Automation)
Continue Reading…

Attaching Tasks to Event in Windows

In my previous blog, we found an error in the Event Log after the scheduled restart of the Windows Task Scheduler task for the Performance Monitor Data Collector Set.

Data collector set SQLAudit failed to start as WORKGROUP\xxxxx$ with error code 0x803000AA.

In this blog, I will outline how we can attach a Task to this event, which will execute a script to start the original task.

Continue Reading…

Using PowerShell to Setup Performance Monitor Data Collector Sets.

Setting up Performance Monitor Data Collector Sets can be a time consuming exercise, especially when you have to do it on multiple servers.
Using sources from multiple places, here is some code I use to create a Data Collector Set, and inside that a number of Data Collectors for Microsoft SQL Server instances running on the server.

Continue Reading…