Trigger an ADF pipeline from another ADF pipeline (using Service principal + certificate) across tenants
So we had a recent requirement of triggering an ADF pipeline from another ADF pipeline across tenants. We cannot use the trigger pipeline activity for this as we are triggering a pipeline in another ADF and we cannot use managed identity because we are triggering an ADF in another tenant.
I think there is a lot of material online on how to achieve this using Service principal and secrets. But of late, our company has been pushing us to use Service principal with certificates instead. Service Principal with certificates is more secure and the reason for this is the asymmetric certificate. There is a public certificate assigned to the Service principal on the Azure AD and the rest API call has the private certificate which is used for authentication. After a bit of struggle I managed to get this working and thought it would be best to document this process and share this with the community.
Most of you may already be aware of how to register an app/service principal in Azure Active directory. Once the app is registered, it can be used to authenticate against AAD and get access to Azure resources. For those of you need a refresher, you can refer to this LINK.
The documentation talks of 2 way to authenticate on AAD.
- Using Client secret
- Using Client certificate
Today we will be using the 2nd method.
the first step is for the client to generate the certificates and share the public certificate with the server. For Dev and test it is fine to use a self signed certificate but for production it is recommended to use a certificate signed by CA.
Execute the following code in Powershell.
New-SelfSignedCertificate -Subject “CN=CertForMyApp” -CertStoreLocation “Cert:\CurrentUser\My” -KeyExportPolicy Exportable -KeySpec Signature
The result should look like this
The next step is to export the certificates and keys.
Search for “mmc” in the windows search box
Click on “File”, “Add/Remove Snap-in”.
Add “Certificates — Current User” to the Selected Snap-ins.
In the Certificate sub-folder under the Personal folder, we should see the new certificate which we had created.
Right click on the certificate, All Tasks, and Export.
Click Next, Next.
Select Base-64 excoded X.509 (.CER) option and click Next.
Select a local location and file name and click Next and Finish to complete the export. The public key will be exported in .CER format. This is the public certificate that needs to be shared with the server and this is what will be uploaded into AAD against the new app registration/service principal.
The public certificate contents will look like this
— — -BEGIN CERTIFICATE — — -
MIIDCDCCAfCgAwIBAgIQOFQ4ssy9KKBB1g8R4Kv8PDANBgkqhkiG9w0BAQUFADAX
MRUwEwYDVQQDDAxDZXJ0Rm9yTXlBcHAwHhcNMjEwODI1MDMwOTM5WhcNMjIwODI1
…….
..
.
— — -END CERTIFICATE — — -
Next we will get the private certificate.
In the mmc application, Right click on the certificate, All Tasks, and Export.
Click Next, Select “Yes Export the Private key” and click Next.
Leave the defaults as shown below and click next.
Then enter a password and click Next.
Select a location and file name and click Next and Finish to complete the export. The private key will be exported in .PFX format.
We will need the base64 encoded version of the pfx private certificate. For this you will need to execute the below section of code.
Clear-Host
Write-Host “This utility will convert a pfx certificate to base64 encoded”
$pfxPath = Read-Host -Prompt ‘Path to your PFX file’
#$fileContentBytes = get-content $pfxPath -Encoding Byte
#[System.Convert]::ToBase64String($fileContentBytes) | Out-File ‘D:\pfx-bytes.txt’
#$b64enc =[System.Convert]::ToBase64String($fileContentBytes)
#Write-Host $b64enc
$password = Read-Host -Prompt ‘password’ -AsSecureString
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$cert.Import($pfxPath, $password, [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::Exportable)
$pfxBlob = [System.Convert]::ToBase64String($cert.Export([Security.Cryptography.X509Certificates.X509ContentType]::Pkcs12))
“THUMBPRINT: $($cert.Thumbprint)”
“PFX BLOB: `n$pfxBlob”
The output will be a long string which we will need a bit later when configuring the web activity on the calling ADF.
Now the target tenant will need to register app/service principal on its Azure AD and upload the public (.CER) certificate onto it.
You can follow the steps in this LINK. Make note of the tenant_id, client_id as you will need this later.
Then click add a certificate or secret.
To upload the public certificate (.cer) format file, click on upload certificate.
Select your public certificate(.cer) file which you had exported earlier.
After uploading the certificate successfully, you should see the screen as below.
Now we need to give the Service principal access to trigger your target ADF pipelines. Go to your target ADF IAM section and assign a contributor role to it.
Next we will create a pipeline to trigger. This is a real simple webactivity pipeline which posts a message to a rest endpoint. To illustrate this, we will use the https://webhook.site/. I use this free site a lot to check if I’m able to make rest API calls successfully. When you go to this site, it will automatically generate an unique endpoint for you as shown below.
Let’s create our triggered pipeline in ADF2 present in AD2. It should look something like this.
I’ve introduced 2 parameters to show that I can even pass some parameters from the calling ADF.
The POST call looks like this.
The url is the webhook endpoint and the body is a concatenation of the 2 parameters.
@concat(pipeline().parameters.var1,pipeline().parameters.var2)
Finally we will create our source pipeline in ADF1 present in AD1.
body : {“var1”:”exampletest1",”var2":”exampletest2"}
Authentication : Service principal
Authentication reference method : Inline
Tenant : Tenant id of the AD2
Service principal ID: Client id of your Service principal that was created on AD2
Service Principal Credential type : Service Principal Certificate
Service principal certificate : The base 64 encoded version of the pfx private certificate (output of your powershell script)
Resource : https://management.azure.com/
Our setup is complete. When we trigger our ADF1 in AD1, it will trigger ADF2 and you should eventually see the result on the webhook.site as shown below.
Hope you find this post useful. Let me know if you have any questions in the comments.
Cheers!