I saw these in the General Activity lists the other day, so I quickly jumped onto the Microsoft documentation, and found there wasn't much about how to use these.
So, I'll outline a short example, along with a link to the ARM template to deploy into your own Data Factory.
Hi All. Staying with the Data Factory V2 theme for this blog. Version 2 introduced a few Iteration & Conditionals activities.
One of these is the Filter activity.
There is a number of use cases for this activity, such as filtering the outputs from the Get Metadata and Lookup Activities.
In this example, I'm going to show you a few ways to apply filtering using the Filter with a few Pipeline parameters. The last example, which I'll also include the ARM template for, is dynamic for both the Operator, and also the item being filtered.
In my previous blog, we set up an Azure Function to return current daylight saving time status for a timezone.
In this blog, we will create a new Version 2 Data Factory, and import an ARM template with a Pipeline to adjust for daylight savings. We can import this template into an existing Data Factory if required.
Apologies for the lack of posts. I will try to post more often!!
Since my last blog where I was meant to follow up and go through Data Factory Custom Activities, Azure Data Factory Version 2 has been released into Preview.
As of today, I still don't believe the 'Triggers' for scheduling a Pipeline allow us to set a timezone, so we have to schedule at UTC.
In this blog, I will go through one of the ways we can work around this whilst creating the triggers in Data Factory itself. (It can also be done through Automation)
In this blog I will give brief instructions to setting up an Azure Batch Service, which is required to execute C# code through Custom Activities via Azure Data Factory.
In my previous blog, we found an error in the Event Log after the scheduled restart of the Windows Task Scheduler task for the Performance Monitor Data Collector Set.
Data collector set SQLAudit failed to start as WORKGROUP\xxxxx$ with error code 0x803000AA.
In this blog, I will outline how we can attach a Task to this event, which will execute a script to start the original task.
In my previous blog, I posted a PowerShell script that can configure a Data Collector Set, with Data Collectors for the server and all MS SQL instances.
As part of my testing, I found that setting up Windows Task Scheduler to restart daily at 5am, that the Perfmon data collector set was not restarting.
Setting up Performance Monitor Data Collector Sets can be a time consuming exercise, especially when you have to do it on multiple servers.
Using sources from multiple places, here is some code I use to create a Data Collector Set, and inside that a number of Data Collectors for Microsoft SQL Server instances running on the server.