normalian blog

Let's talk about Microsoft Azure, ASP.NET and Java!

How to dig into API Management performance with Application Insights

As you know, Azure API Management is integrated with API management like below.
docs.microsoft.com
This article describe how to setup the integration and utilize the feature.

Create your Application Insights

You need to create Application Insights to associate with API management like below. Note you need to choose "General" as Application Type.
f:id:waritohutsu:20180726041037p:plain

Associate your Application Insights with API Management and configure it

Choose "Application Insight" item from left side menus of API Management and associate it like below.
f:id:waritohutsu:20180726041210p:plain

Next, choose "APIs" item from left side menus of API Management and click "Settings" tab. Change "Sampling" and "First bytes of body(max 1024)" after enabling Application Insights like below.
f:id:waritohutsu:20180726041520p:plain
Change "Sampling" value as 100 to pick up all request into Application Insight. And change "First bytes of body(max 1024)" as 1024 or your demand if you need to confirm request body.

How to confirm in Application Insights

At first, you can use "Live Metrics Stream". This shows Request duration, CPU usage, committed memory and others like below.
f:id:waritohutsu:20180726042009p:plain

Second, you can dig into from "performance" tab like below. You can confirm who many requests, how request duration and others. And you can also dig into requests to confirm their dependencies.
f:id:waritohutsu:20180726042125p:plain
f:id:waritohutsu:20180726042134p:plain

Configuration tips when you need to upgrade Service Fabric cluster applications

Service Fabric is one of components to offer Microservice architecture and it's also used with CI/CD tools such like VSTS. In this post, you can learn tips to construct Microservice architecture CI/CD pipeline.

Error #1 "You must first remove the existing application before a new application can be deployed or provide a new name for the application."

You will get this error when you try to upgrade your Service Fabric applications as latest version. Here is error message within VSTS release process.

2018-07-24T23:02:08.4239940Z Imported cluster client certificate with thumbprint 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'.
2018-07-24T23:02:27.2235680Z Successfully connected to cluster.
2018-07-24T23:02:27.2624467Z Searching for path: D:\a\r1\a
2018-07-24T23:02:27.3693528Z No items were found with search pattern D:\a\r1\a.
2018-07-24T23:02:35.2334317Z ##[error]An application with name 'fabric:/MySFASPAppType' already exists, its type is 'MySFASPAppType' and version is '1.0.0.20180724.1'. You must first remove the existing application before a new application can be deployed or provide a new name for the application.
2018-07-24T23:02:35.8774898Z ##[section]Finishing: Deploy Service Fabric Application

You need to configure upgrade option for Service Fabric cluster with Visual Studio like below. You can come up this dialog by right click and choose "Publish".
f:id:waritohutsu:20180725161131p:plain

The configuration will be reflected into "your Service Fabric project name"\PublishProfiles\Cloud.xml like below.

<?xml version="1.0" encoding="utf-8"?>
<PublishProfile xmlns="http://schemas.microsoft.com/2015/05/fabrictools">
  <ClusterConnectionParameters ConnectionEndpoint="" />
  <ApplicationParameterFile Path="..\ApplicationParameters\Cloud.xml" />
  <CopyPackageParameters CompressPackage="true" />
  <UpgradeDeployment Mode="UnmonitoredAuto" Enabled="true">
    <Parameters UpgradeReplicaSetCheckTimeoutSec="1" Force="True" />
  </UpgradeDeployment>
</PublishProfile>

Confirm "UpgradeDeployment" tag including its child tags. This should solve the error.

Error #2 "The content in ConfigPackage Name:Config and Version:x.x.x in Service Manifest 'xxxxxxxxxxxxxxx' has changed, but the version number is the same."

You should look over - Start-ServiceFabricApplicationUpgrade (ServiceFabric) | Microsoft Docs before proceeding this post. Service Fabric applications have several versions such like Service Fabric package itself, code and config. You need to update some of the versions to upgrade Service Fabric applications.
A temporary solution is to use "Build.BuildId" for the versions. Edit ApplicationManifest.xml and ServiceManifest.xml in your Service Fabric project like below.

"your Service Fabric project name"\ApplicationPackageRoot\ApplicationManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest ApplicationTypeName="SFwithASPNetAppType"
                     ApplicationTypeVersion="1.0.#{Build.BuildId}#"
                     xmlns="http://schemas.microsoft.com/2011/01/fabric"
                     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Parameters>
    <Parameter Name="GuestContainer1_InstanceCount" DefaultValue="-1" />
  </Parameters>
  <!-- Import the ServiceManifest from the ServicePackage. The ServiceManifestName and ServiceManifestVersion 
       should match the Name and Version attributes of the ServiceManifest element defined in the 
       ServiceManifest.xml file. -->
  <ServiceManifestImport>
    <ServiceManifestRef ServiceManifestName="GuestContainer1Pkg" ServiceManifestVersion="1.0.#{Build.BuildId}#" />
    <ConfigOverrides />
    <Policies>
      <ContainerHostPolicies CodePackageRef="Code" Isolation="hyperv">
    ...

"your Service Fabric project name"\ApplicationPackageRoot\"ServiceManifestName"\ServiceManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest Name="GuestContainer1Pkg"
                 Version="1.0.#{Build.BuildId}#"
                 xmlns="http://schemas.microsoft.com/2011/01/fabric"
                 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ServiceTypes>
    <!-- This is the name of your ServiceType.
         The UseImplicitHost attribute indicates this is a guest service. -->
    <StatelessServiceType ServiceTypeName="GuestContainer1Type" UseImplicitHost="true" />
  </ServiceTypes>

  <!-- Code package is your service executable. -->
  <CodePackage Name="Code" Version="1.0.#{Build.BuildId}#">
    <EntryPoint>
      <ContainerHost>
        <ImageName>"my container registry account ".azurecr.io/#{Build.Repository.Name}#:#{Build.BuildId}#</ImageName>
      </ContainerHost>
    </EntryPoint>
    <EnvironmentVariables>
      <EnvironmentVariable Name="VariableName" Value="VariableValue"/>
    </EnvironmentVariables>
  </CodePackage>

  <ConfigPackage Name="Config" Version="1.0.#{Build.BuildId}#" />

  <Resources>
    <Endpoints>
      <Endpoint Name="GuestContainer1TypeEndpoint" UriScheme="http" Port="xxxx" Protocol="http" />
    </Endpoints>
  </Resources>
</ServiceManifest>

You also need to follow Replace configuration files with environment variables on VSTS tasks - normalian blog to replace #{Build.BuildId}# in your xml files.

Embed Jenkins portal into Visual Studio Team Services dashboard

As you know, lots of developers are using Jenkins for their CI/CD pipeline mainly for Java and other OSS developments. But some of such developers also use Visual Studio Team Services called VSTS for .NET development. Of course, we can develop both .NET, Java and other OSS even in VSTS, but many development fields have existing Jenkins pipelines as their assets.
It's difficult to migrate their Jenkins pipeline into VSTS In such a case, but we can easily embed your Jenkins portal into VSTS dashboard. We can collaborate both VSTS and Jenkins using by such a feature. In this article, you can learn how to setup that!

Jenkins Setup - if you need

This step isn't needed if you have already setup Jenkins in your environment. Refer to contents below if you want to setup it on Microsoft Azure.

Install XFrame Filter Plugin into Jenkin and enable to use iFrame

Install a plugin called "XFrame Filter Plugin" into your Jenkins, because it needs to enable iFrame to embed your Jenkins portal into VSTS dashboard.
Go to your Jenkins portal and choose "Manage Jekins" - "Manage Plugins" like below.
f:id:waritohutsu:20180713224159p:plain
Next, click "Available" and input "XFrame" to find "XFrame Filter Plugin". You can install the plugin easily just enable checkbox and click "Downlaod now and install after restart".

After completion of the install, you need to configure the plugin. Go to your Jenkins portal again and choose "Manage Jekins" - "Configure System" like below.
f:id:waritohutsu:20180713224703p:plain

Find the plugin among them, enable the feature and input your VSTS account URL into "X-Frame-Options Options" box like below.
f:id:waritohutsu:20180713224949p:plain

Embed Jenkins portal using by “Embedded Webpage” into VSTS Dashboard

Next, you need to go to your VSTS dashboard and add "Embedd Webpage" like below,
f:id:waritohutsu:20180713230126p:plain
Configure "Embedd Webpage" to input your Jenkins URL like below.
f:id:waritohutsu:20180713230409p:plain

I believe your browser doesn't trust your Jenkins URL. so you also need to enable untrusted contents like below,
f:id:waritohutsu:20180713230626p:plain

Finally, you can watch Jenkins portal on VSTS dashboard, so you can watch both VSTS and Jenkins pipeline like below.
f:id:waritohutsu:20180713230803p:plain

How to execute PowerShell scripts inside Azure VMs from external

There are some ways to execute PowerShell scripts inside Azure VMs such like PowerShell remoting. Recently, it comes up cool feature to execute scripts inside Azure VMs easily. This article introduces how to manage that.

Execute PowerShell scripts inside Azure VMs from Azure Portal

Go to Azure Portal and choose an Azure VM among your VMs, so you can find "Run command" menu from left menus. Next, choose "RunPowerShellScript" menu, so you can execute PowerShell scripts like below.
f:id:waritohutsu:20180627100025p:plain
I have already located a text file into "F;\temp\hello.txt" path on Azure VM before executing above scripts to take this screenshot. This means you can manage files inside Azure VMs.
Here is diagram for this scenario. We send HTTP requests as REST API call to VM Agent and the agent execute your PowerShell scripts inside the VM.
f:id:waritohutsu:20180628041811p:plain

Execute PowerShell scripts inside Azure VMs from client machines

You can execute the scripts with PowerShell command named Invoke-AzureRmVMRunCommand. Here is diagram for this scenario. f:id:waritohutsu:20180628042618p:plain
Follow commands snippet below in your local machine PowerShell ISE to execute your scripts inside VM.

$rgname = 'your vm resource group'
$vmname = 'your vm name'
$localmachineScript = 'PowerShell script file on your local machine like script-test.ps1'
Invoke-AzureRmVMRunCommand -ResourceGroupName $rgname -Name $vmname -CommandId 'RunPowerShellScript' -ScriptPath $localmachineScript -Parameter @{"arg1" = "var1";"arg2" = "var2"} -Debug 

Confirm your Azure PowerShell module version and authentication if the script doesn't work well.

Execute PowerShell scripts inside Azure VMs from Azure Automation

At first, update Azure Automation modules for your Azure Automation account. Go to your Azure Automation account and choose "Modules" menu from left menu and click "Update Azure Modules" to be latest versions like below.
f:id:waritohutsu:20180627102437p:plain

Next, It's needed to locate your script into downloadable place such like Azure Storage, because we can't locate any files into Azure Automation runtime environment. In this case, I have located a script file into "https://change-your-storage-account-name.blob.core.windows.net/scripts/script-test.ps1" and the content is same with above. Here is diagram for this scenario.
f:id:waritohutsu:20180628043548p:plain

Finally, create a Runbook for the script like below and execute it. This need to authenticate to Azure AD.

$connection = Get-AutomationConnection -Name "AzureRunAsConnection"
Write-Output $connection
Add-AzureRMAccount -ServicePrincipal -Tenant $connection.TenantID -ApplicationId $connection.ApplicationID -CertificateThumbprint $connection.CertificateThumbprint

$rgname = 'your vm resource group'
$vmname = 'your vm name'
$localmachineScript = 'PowerShell script file on your local machine like script-test.ps1'
wget "https://automationbackupstorage.blob.core.windows.net/scripts/$localmachineScript" -outfile $localmachineScript 
Invoke-AzureRmVMRunCommand -ResourceGroupName $rgname -Name $vmname -CommandId 'RunPowerShellScript' -ScriptPath $localmachineScript -Parameter @{"arg1" = "var1";"arg2" = "var2"} -Debug 

How to handle exceptions for the scripts inside Azure VMs

It's needed to handle errors when you will integrate this scripts execution into your workflow. I have updated 'script-test.ps1' script like below,

cd F:\temp
type hello.txt
throw "Error trying to do a task @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@"

Here is a result of the script execution.

PS D:\temp> $rgname = 'your vm resource group'
$vmname = 'your vm name'
$localmachineScript = 'script-test.ps1'
$result = Invoke-AzureRmVMRunCommand -ResourceGroupName $rgname -Name $vmname -CommandId 'RunPowerShellScript' -ScriptPath $localmachineScript -Parameter @{"arg1" = "var1";"arg2" = "var2"} -Debug 
DEBUG: 6:28:11 PM - InvokeAzureRmVMRunCommand begin processing with ParameterSet 'DefaultParameter'.

...

DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
...

Headers:
x-ms-client-request-id        : f0edfe29-5abf-4d7f-9d83-8c98b3e59891
accept-language               : en-US

Body:
{
  "commandId": "RunPowerShellScript",
  "script": [
    "cd F:\\temp",
    "type hello.txt",
    "throw \"Error trying to do a task @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\""
  ],
  "parameters": [
    {
      "name": "arg1",
      "value": "var1"
    },
    {
      "name": "arg2",
      "value": "var2"
    }
  ]
}


DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

...

Body:
{
  "startTime": "2018-06-26T18:28:14.5508701-07:00",
  "endTime": "2018-06-26T18:28:36.3646186-07:00",
  "status": "Failed",
  "error": {
    "code": "VMExtensionProvisioningError",
    "message": "VM has reported a failure when processing extension 'RunCommandWindows'. Error message: \"Finished executing command\"."
  },
  "name": "bc901040-54ad-4ff1-a8ee-c9794b7a34cb"
}


DEBUG: AzureQoSEvent: CommandName - Invoke-AzureRmVMRunCommand; IsSuccess - True; Duration - 00:00:33.0677753; Exception - ;
DEBUG: Finish sending metric.
DEBUG: 6:28:45 PM - InvokeAzureRmVMRunCommand end processing.
DEBUG: 6:28:45 PM - InvokeAzureRmVMRunCommand end processing.
Invoke-AzureRmVMRunCommand : Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure when processing extension 'RunCommandWindows'. Error message: 
"Finished executing command".'
ErrorCode: VMExtensionProvisioningError
ErrorMessage: VM has reported a failure when processing extension 'RunCommandWindows'. Error message: "Finished executing command".
StartTime: 6/26/2018 6:28:14 PM
EndTime: 6/26/2018 6:28:36 PM
OperationID: bc901040-54ad-4ff1-a8ee-c9794b7a34cb
Status: Failed
At line:1 char:1
+ Invoke-AzureRmVMRunCommand -ResourceGroupName $rgname -Name $vmname - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Invoke-AzureRmVMRunCommand], ComputeCloudException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.Automation.InvokeAzureRmVMRunCommand
 
DEBUG: AzureQoSEvent: CommandName - Invoke-AzureRmVMRunCommand; IsSuccess - False; Duration - 00:00:33.0677753; Exception - Microsoft.Azure.Commands.Compute.Common.ComputeCloudException: Long
 running operation failed with status 'Failed'. Additional Info:'VM has reported a failure when processing extension 'RunCommandWindows'. Error message: "Finished executing command".'
ErrorCode: VMExtensionProvisioningError
ErrorMessage: VM has reported a failure when processing extension 'RunCommandWindows'. Error message: "Finished executing command".
StartTime: 6/26/2018 6:28:14 PM
EndTime: 6/26/2018 6:28:36 PM
OperationID: bc901040-54ad-4ff1-a8ee-c9794b7a34cb
Status: Failed ---> Microsoft.Rest.Azure.CloudException: Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure when processing extension 'RunCommandWi
ndows'. Error message: "Finished executing command".'
   at Microsoft.Rest.ClientRuntime.Azure.LRO.AzureLRO`2.CheckForErrors()
   at Microsoft.Rest.ClientRuntime.Azure.LRO.AzureLRO`2.<StartPollingAsync>d__17.MoveNext()
...
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.Compute.VirtualMachinesOperationsExtensions.RunCommand(IVirtualMachinesOperations operations, String resourceGroupName, String vmName, RunCommandInput paramet
ers)
   at Microsoft.Azure.Commands.Compute.Automation.InvokeAzureRmVMRunCommand.<ExecuteCmdlet>b__0_0()
   at Microsoft.Azure.Commands.Compute.ComputeClientBaseCmdlet.ExecuteClientAction(Action action)
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Commands.Compute.ComputeClientBaseCmdlet.ExecuteClientAction(Action action)
   at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord();
DEBUG: Finish sending metric.
DEBUG: 6:28:47 PM - InvokeAzureRmVMRunCommand end processing.
DEBUG: 6:28:47 PM - InvokeAzureRmVMRunCommand end processing.

PS D:\temp> $result

It seems to be difficult to handle the errors, because there are no contents including message of "Error trying to do a task @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@". In this case, -ErrorVariable option should be useful. Update the script again and execute it like below.

PS D:\temp> $rgname = 'your vm resource group'
$vmname = 'your vm name'
$localmachineScript = 'script-test.ps1'
Invoke-AzureRmVMRunCommand -ResourceGroupName $rgname -Name $vmname -CommandId 'RunPowerShellScript' -ScriptPath $localmachineScript -Parameter @{"arg1" = "var1";"arg2" = "var2"} -ErrorVariable result
echo "============================="
$result.Count
echo "============================="
$result
echo "============================="
$result[1]

Invoke-AzureRmVMRunCommand : Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure when processing extension 'RunCommandWindows'. Error message: 
"Finished executing command".'
ErrorCode: VMExtensionProvisioningError
ErrorMessage: VM has reported a failure when processing extension 'RunCommandWindows'. Error message: "Finished executing command".
StartTime: 6/26/2018 6:35:02 PM
EndTime: 6/26/2018 6:35:17 PM
OperationID: 2ea46d42-2523-4f23-9135-9a595f62f656
Status: Failed
At line:1 char:1
+ Invoke-AzureRmVMRunCommand -ResourceGroupName $rgname -Name $vmname - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Invoke-AzureRmVMRunCommand], ComputeCloudException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.Automation.InvokeAzureRmVMRunCommand
 
=============================
1
=============================
Invoke-AzureRmVMRunCommand : Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure when processing extension 'RunCommandWindows'. Error message: 
"Finished executing command".'
ErrorCode: VMExtensionProvisioningError
ErrorMessage: VM has reported a failure when processing extension 'RunCommandWindows'. Error message: "Finished executing command".
StartTime: 6/26/2018 6:35:02 PM
EndTime: 6/26/2018 6:35:17 PM
OperationID: 2ea46d42-2523-4f23-9135-9a595f62f656
Status: Failed
At line:1 char:1
+ Invoke-AzureRmVMRunCommand -ResourceGroupName $rgname -Name $vmname - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Invoke-AzureRmVMRunCommand], ComputeCloudException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.Automation.InvokeAzureRmVMRunCommand
 
=============================

PS D:\temp> $result

Unfortunately, we can't take error messages from inside PowerShell scripts but we can find there are errors or not. We should use both Azure Automation logs and logs inside Azure VMs.

How to solve RDP access error "CredSSP Encryption Oracle Remediation"

With the release of the March 2018 Security bulletin, it causes some changes. When you try to connect to updated VMs, you probably get error like below.
f:id:waritohutsu:20180514044358p:plain

[Window Title]
Remote Desktop Connection

[Content]
An authentication error has occurred. The function requested is not supported

Remote computer: 13.93.225.149
This could be due to CredSSP encryption oracle remediation.
For more information, see https://go.microsoft.com/fwlink/?linkid=866660

[OK]

Refer to Unable to RDP to Virtual Machine: CredSSP Encryption Oracle Remediation – Azure Virtual Machines to know detail. In this article, I will describe how to solve this issue simply.

Execute gpedit.msc and go to Computer Configuration / Administrative Templates / System / Credentials Delegation like below.
f:id:waritohutsu:20180514044816p:plain

Change Encryption Oracle Remediation policy to Enabled, and Protection Level to Vulnerable like below.
f:id:waritohutsu:20180514044941p:plain

API Management and Service Fabric Collaboration for Global Scale Applications

As you know, you can achieve Microservice architecture by using Service Fabric, but you might want to need request routing features for your applications for multi languages, cross devices or others. In such a case, you can use API Management for it. In this article, you can learn how to setup API Management with Service Fabric.

Edit ServiceManifest.xml of your Serviec Fabric project

At first, make a REST API application and deploy it into your Service Fabric cluster. Note to edit "ServiceManifest.xml" file in your Service Fabric project not to specify actual port like below. This setup is needed to collaborate API Management and Service Fabric.

<Resources>
  <Endpoints>
    <Endpoint Protocol="http" Name="ServiceEndpoint" Type="Input" />
  </Endpoints>
</Resources>

Download certificate file to access Service Fabric cluster

You probably created the certificate automatically when you made your Service Fabric cluster. Go to your Service Fabric cluster, choose "security" tab and pick up the certificate thumbprint like below.
f:id:waritohutsu:20180415080735p:plain

Next, download the certificate file as pfx into your machine. Go to KeyVault, choose "certificate" tab, select your certificate and choose "Download in PFX/PEM format" like below.
f:id:waritohutsu:20180415081024p:plain

Save thumbprint and pfx file to use ARM Template in later section.

Deploy new API Management instance by using ARM Template

Download apim.json and apim.parameters.json ARM Templates from service-fabric-api-management/apim.json at master · Azure-Samples/service-fabric-api-management · GitHub. And add '"validateCertificateChain": false into apim.json' if you will use self-signed certificate file like below .

            "apiVersion": "2017-03-01",
            "type": "Microsoft.ApiManagement/service/backends",
            "name": "[concat(parameters('apimInstanceName'), '/', parameters('service_fabric_backend_name'))]",
            "dependsOn": [
                "[resourceId('Microsoft.ApiManagement/service', parameters('apimInstanceName'))]",
                "[resourceId('Microsoft.ApiManagement/service/certificates', parameters('apimInstanceName'), parameters('serviceFabricCertificateName'))]"
            ],
            "properties": {
                "description": "My Service Fabric backend",
                "url": "fabric:/fake/service",
                "protocol": "http",
                "resourceId": "[parameters('clusterHttpManagementEndpoint')]",
                "tls":{
                    "validateCertificateChain": false
                },
                "properties": {
                    "serviceFabricCluster": {
                        "managementEndpoints": [
                            "[parameters('clusterHttpManagementEndpoint')]"
                        ],
                        "clientCertificateThumbprint": "[parameters('serviceFabricCertificateThumbprint')]",
                        "serverCertificateThumbprints": [
                            "[parameters('serviceFabricCertificateThumbprint')]"
                        ],
                        "maxPartitionResolutionRetries": 5
                    }
                }
            }
        },

Update apim.parameters.json like below.

{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "apimInstanceName": {
            "value": "sfapim01"
        },
        "subnetName": {
            "value": "API-Subnet"
        },
        "apimPublisherEmail": {
            "value": "mymail@address.com"
        },
        "apimSku": {
            "value": "Developer"
        },
        "serviceFabricCertificateName": {
            "value": "Daichi Isami"
        },
        "serviceFabricCertificate": {
            "value": "base64 encoded string of your pfx file. don't insert breaklines"
        },
        "certificatePassword": {
            "value": ""
        },
        "serviceFabricCertificateThumbprint": {
            "value": "your Cluster certificates thumbprint"
        },
        "url_path": {
            "value": "/api/values"
        },
        "clusterHttpManagementEndpoint": {
            "value": "https://'your cluster name'.westus.cloudapp.azure.com:19080"
        },
        "inbound_policy":{
            "value": "<policies>\r\n  <inbound>\r\n    <base />\r\n    <set-backend-service backend-id=\"servicefabric\" sf-service-instance-name=\"fabric:/SFApiApp/Web1\" sf-resolve-condition=\"@((int)context.Response.StatusCode != 200)\" />\r\n  </inbound>\r\n  <backend>\r\n    <base />\r\n  </backend>\r\n  <outbound>\r\n    <base />\r\n  </outbound>\r\n  <on-error>\r\n    <base />\r\n  </on-error>\r\n</policies>"
        },
        "policies_policy_name": {
            "value": "policy"
        },
        "apis_service_fabric_app_name": {
            "value": "service-fabric-app"
        },
        "apim_service_fabric_product_name": {
            "value": "service-fabric-api-product"
        },
        "service_fabric_backend_name": {
            "value": "servicefabric"
        },
        "apis_service_fabric_app_name_operation": {
            "value": "service-fabric-app-operation"
        },
        "vnetName": {
            "value": "VNet-sf-sample01-1709cluster"
        },
        "vnetVersion": {
            "value": "2017-03-01"
        },
        "networkSecurityGroupName": {
            "value": "apim-vnet-security-03"
        },
        "networkSecurityGroupVersion": {
            "value": "2017-03-01"
        }
    }

You can put blank for certificatePassword value if you created your certificate file automatically. Refer to commands for base64encode for your certificate below if you need.

$bytes = [System.IO.File]::ReadAllBytes("C:\temp\yourpfxfile.pfx")
$b64 = [System.Convert]::ToBase64String($bytes);
$b64 

It should takes 30 or 40 minutes to complete this deployment.

Access your Service Fabric application via API Management

Go to "Developer Portal" of your API Management, choose "Service Fabric App" among APIs and click "Try it" button. Now, you can send requests to your API application via API Management like below.
f:id:waritohutsu:20180415082448p:plain

Tips No.1: Troubleshoot - "Service Fabric exception when trying to resolve partition: A Security error has occurred, failed to verify remote certificate"

You might get error messages below if you use self-signed certificate file.

service-fabric-backend (1371 ms)
{
    "message": "Service Fabric exception when trying to resolve partition: A Security error has occurred, failed to verify remote certificate.",
    "serviceName": {},
    "resourceId": "https://sf-sample01-1709cluster.westus.cloudapp.azure.com:19080",
    "managementEndpoint": [
        "https://sf-sample01-1709cluster.westus.cloudapp.azure.com:19080"
    ]
}

You should forget to update apim.json. Refer to " Deploy new API Management instance by using ARM Template" section in this article.

Tips No.2: Don't use “Client certificates” for API Management

As you know, Service Fabric uses multiple certificates for itself. Note that use "Cluster certificates" not "Client certificates" for API Management.
f:id:waritohutsu:20180415110457p:plain

How to setup CI/CD pipeline with Service Fabric, VSTS and Windows Container

We have tried lots of features to collaborate wtih Service Fabric, VSTS and Docker containers. I have realized it's needed to describe overview of the architecture, so you can learn it following this article.

Overview of Service Fabric, VSTS and Windows Container architecture

At first, refer to the architecture diagram below. Purple lines mean processing flows of the CI/CD pipeline and green lines mean its setting dependencies.
f:id:waritohutsu:20180411082506p:plain

Create below resources to setup this architecture.

  • Service Fabric cluster
  • VSTS Proejct, Build Process and Release Process
  • Azure Container Registry
  • Virtual Machines for VSTS Private Agent

In this article, you can find references to Service Fabric cluster, VSTS and Virtula Machines. But create Azure Container Registry for yourself and it should be quite easy.

a - Setup Private Agent for VSTS Build Definitions

Unfortunately, Windows Docker base image sizes are about 1.5G. It takes much time to download and build Docker images if you don't use Pirvate Agent. By caching the Docker images, its building time can be largely reduced.

3. and 4. Deploy deployment artifacts into Service Fabric cluster and download your Docker images from Azure Container Registry