normalian blog

Let's talk about Microsoft Azure, ASP.NET and Java!

Object Replication - easiest way to replicate Block BLOBs into other regions?

Object Replication is a new feature for Azure Storage. This feature enable you to transfer BLOB objects into different regions easily with minimizing latency. You might know Data redundancy - Azure Storage | Microsoft Docs, but this feature is possible to replicate your BLOBs to only your paired region. In addition that, it's a little bit tricky to reach out your data on paired regions.

Object Replication offers feature to replicate your Block BLOBs to containers in any regions with just few setting on your Storage accounts, so this feature should be quite useful to replicate your data across countries. I believe most readers of this article would be quite busy, so here are summaries for Object Replication at this time - please note this Object Replication is under preview on June 2020 now.

  • This feature is for Block BLOBs, so you can't utilize this feature for VHD files, namely, Page BLOB
  • It takes about 2 minutes to transfer BLOB objects regardless regions but it would depends on size.
  • Need to set source containers "Public Access Level" as "Container" or "Blob", and this means it's not possible to use this feature with "Private". update on 6/14/2020
  • Enable to utilize all "Public Access Level" as "Private", "Container" or "Blob"
  • Need to setup as container level on Azure Storage accounts to replicate. You can setup up to two outbound policies per Azure Storage account
  • Available only France Central, Canada East and Canada Central on June 2020 now.
  • Pricing Tier, "Hot" "Cool" and "Archive", won't be propagated accurately. Refer to a section below. - update on 6/14/2020

How to enable Object Replication on your subscription

Follow this article at first. You need to register a couple of resource providers because this feature depends on other features such as Change feed and Versioning. Just a reference, it took about a week to enable Object Replication on my subscription.

Setup and try Object Replication on your Storage accounts

After provision of Object Replication on your subscription, you can find menu of Object Replication on your storage accounts like below. You can confirm both destination and source accounts at once.
f:id:waritohutsu:20200608025444p:plain

You can setup policies with specifying containers, Filters and "Copy over". It's also possible to handle objects which should be copied into other accounts.
f:id:waritohutsu:20200608030439p:plain

As you can confirm below, it takes about 1 minutes with less than 1MB file to replicate.
f:id:waritohutsu:20200608031250p:plain

Today, it's possible to setup up to two outbound policies per Storage accounts like below.
f:id:waritohutsu:20200608031718p:plain

Pricing Tier propagation

I have tried three types tire cases.

  • Red Box: Upload a file as "Hot" tier at first and change the tier into "Archive"
  • Green Box: Upload a file as "Archive" tier
  • Blue Box: Upload a file as "Cool" tier

f:id:waritohutsu:20200615033736p:plain

As you can confirm with the screenshot, here are results.

  • Red Box: Upload a file as "Hot" tier at first and change the tier into "Archive" -> pricing tier isn't propagated into dest blob
  • Green Box: Upload a file as "Archive" tier -> The blobs won't be copyed into dest containers
  • Blue Box: Upload a file as "Cool" tier -> "Cool" tier blobs will be copied into dest containers as "Hot" tier.

Azure VMs cost reduction tips for dev and test environment

I believe Azure VMs is the most popular feature for all Azure users, and Azure VM usage would occupy most charge among your Azure billing. You will require high performance VM at the beginning because it would be needed to setup something to build up your development or test enviroments, but such requirements are not so much after the setup. There are two good options to offer good tips for your wallet-friendly.

  • Choose B-Series type for Azure VMs
  • Change disk type from Premium to Standard when your VMs are deallocated

Keep in mind that don't adopt this concept into your production envrionments.

Choose B-Series type for Azure VMs

I believe no need to explain too much about this topic. B-Serise is burstable instances. This type of instances offers a quite good cost effective way to utilize Azure VMs.
https://azure.microsoft.com/en-us/blog/introducing-b-series-our-new-burstable-vm-size/

Change disk type from Premium to Standard when your VMs are deallocated

This is a little bit tricky than just choosing B-Series VMs. You might misundersand that you can't change Disk types after attaching your disks to your Azure VMs. It's partially true because it's not possible to change Disk Types when your Azure VMs are running like below.
f:id:waritohutsu:20200607074035p:plain

But you can find quite interesting description in red box on this image. You can change your disk types when your Azure VMs are deallocated like below.
f:id:waritohutsu:20200607074322p:plain

You can choose three disk types - "Premium SSD", "Standard SSD" or "Standard HDD". What's the pro-con for them? You can confirm details both performance and pricing perspectives by referring articles below.

"Premium SSD" has much better IOPS than cheapest type "Standard HDD" but the price is almost three times. In addition this, test and development environments won't be utilized so much IOPS in most of cases. You should acquire quite good cost reduction by following this tips.

Tips to utilize Windows Server containers on AKS

Microsoft has announced that Azure Kubernetes Service (AKS) supports Windows Server containers as GA. This is quite useful and essential feature to containerize your ASP.NET Framework applications. In this article, you can acquire tiny tips to utilize Windows Server containers on AKS.

Enable Azure CNI (advanced) for Windows Server Container

Note that AKS requires to enable " Azure CNI (advanced) network plugin" to utilize Windows Server Containers. Choose "Advanced" as Network configuration like below when you try to create AKS clusters.
f:id:waritohutsu:20200511055511p:plain

You can confirm your AKS clusters are enabled Azure CNI on Azure Portal.
f:id:waritohutsu:20200511055548p:plain

Next, you need to create node pools as Windows OS type to deploy your Windows Server Container applications like below.
f:id:waritohutsu:20200511055835p:plain

Windows Server Container size

Windows Server Container requires huge capacity than Linux images. I have just pushed a simple hello world ASP.NET application into my Azure Container Repository(ACR) but it uses 1.08GB on my ACR. It will take a much time to upload your container images first time, so please note your network bandwidth not only ACR capacities when you push your container images into ACRs.
f:id:waritohutsu:20200511060031p:plain

Manage authorization for your application with user account attributes

Azure AD offers quite useful features to manage accessibilities for your applications. I believe most Azure developers has already utilized user groups to assign privilege easily, but I guess many people don’t know “Dynamic User” user group. This user group enable to authorize users with user account attributes.

Let's setup to manage accessibilities with job title by using Dynamic User group. Here are accounts which are verified by the group.
f:id:waritohutsu:20200509043431p:plain
f:id:waritohutsu:20200509043445p:plain

How to create Dynamic User group

Let’s go to Azure Portal, choose Azure Active Directory, and click to “New Group” at first .
f:id:waritohutsu:20200509043119p:plain

You can choose “Dynamic User” as membership type like below.
f:id:waritohutsu:20200509043133p:plain

Click “Add dynamic query” to setup query to authorize users. This sample authorize users who contain “Principal” for their job title. It’s also possible to create complex queries to meet your business requirements.
f:id:waritohutsu:20200509043143p:plain

Click “Validate Rules (Preview)” like below. You can confirm your queries will works well or not.
f:id:waritohutsu:20200509043152p:plain

Reduce AKS clusters cost by setup zero node count for user mode node pools

Here is interesting article - Release Release 2020-04-13 · Azure/AKS · GitHub. You can find that "AKS now allows User nodepools to scale to 0" in the article. This feature enables to reduce AKS cost in your environments. I believe you would try to change node count by using az command, but it won't work well at this time - 5/1/2020. Please note this setting is possible for only User mode node pools not System mode.

$subcriptionId = "YOUR SUBSCRIPTION ID"
$rg = "YOUR RESOURCE GROUP"
$clustername = "YOUR AKS CLUSTER NAME"
$poolname = "YOUR NODE POOL NAME"
$count = 0
az aks scale --resource-group $rg --name $clustername --node-count $count --nodepool-name $poolname

f:id:waritohutsu:20200502043921p:plain

This issue is caused that az command doesn't support to setup zero node count for user mode node pools at this time. There are two options to achieve this setting here.

Change node count on https://resources.azure.com/

Open https://resources.azure.com/ and find your user mode node pool of your AKS clusters. Put "Edit" button to enable to change Azure resources setting and edit value of "count" as zero.
f:id:waritohutsu:20200502044622p:plain

Please note this setting is possible only User mode node pools. It will fail to change node count into zero for System mode node pools.
f:id:waritohutsu:20200502044934p:plain

Use REST API to change node count

You can REST API by using az command. Here is example to setup zero node count for user mode node pools.

$subcriptionId = "YOUR SUBSCRIPTION ID"
$rg = "YOUR RESOURCE GROUP"
$clustername = "YOUR AKS CLUSTER NAME"
$poolname = "YOUR NODE POOL NAME"
$count = 0

$body = "{  \`"properties\`": {    \`"count\`": ${count} } }"
$header = "{\`"Content-Type\`": \`"application/json\`"}"
az rest -u "https://management.azure.com/subscriptions/${subcriptionId}/resourceGroups/${rg}/providers/Microsoft.ContainerService/managedClusters/${clustername}/agentPools/${poolname}?api-version=2020-03-01" --method put --headers $header --body $body

You can confirm this setting on Azure Portal.
f:id:waritohutsu:20200502045455p:plain

Spot node pool limitation for AKS

Azure Kubernetes Service(AKS) recently offers spot node pool feature as preview - 4/30/2020 right now. This feature enable Azure developers to reduce VM costs by using spot VMs for AKS clusters.
Refer to Preview - Add a spot node pool to an Azure Kubernetes Service (AKS) cluster - Azure Kubernetes Service | Microsoft Docs. Here is a sample command to execute on Azure Portal, but you need to enable this preview feature by following the article before running commands below. You can execute this command successfully.

az aks nodepool add \
    --resource-group YOUR-RESOURCE-GROUP \
    --cluster-name YOUR-AKS-CLUSTER-NAME \
    --name spotnode01 \
    --priority Spot \
    --node-vm-size Standard_DS2_v2  \
    --node-count 1 \
    --eviction-policy Delete \
    --spot-max-price -1 \
    --no-wait

I guess you also want to save cost by using burstable instances named "B series". You can execute a command to specify B series instances but it will be failed .

az aks nodepool add \
    --resource-group YOUR-RESOURCE-GROUP \
    --cluster-name YOUR-AKS-CLUSTER-NAME \
    --name spotnode02 \
    --priority Spot \
    --node-vm-size Standard_B2ms \
    --node-count 1 \
    --eviction-policy Delete \
    --spot-max-price -1 \
    --no-wait

Here is a screenshot for commands above. You can execute commands to add node pools by using spot VMS, but provisions will fail.
f:id:waritohutsu:20200501144554p:plain

This is limitation of spot VMs. Refer to Use Azure Spot VMs - Azure Windows Virtual Machines | Microsoft Docs. B-series and Promo versions of any size (like Dv2, NV, NC, H promo sizes) are not supported at this time.

Upload Camera images into Azure Blob Storage by PowerApps

As you know, PowerApps offers a bunch of useful features to build up powerful applications easily. It's also possible to retrieve Microsoft Azure Platform not only Power Platform. You can acquire knowledge how upload images token by Camera into Azure Blob Storage. Here are steps to build up it.

  1. Azure Storage setup on Azure Portal
  2. Create Connection for Azure Storage on PowerApps Studio for Web
  3. Create apps by using the Connection on PowerApps Studio for Web

Azure Storage setup on Azure Portal

At first, open Azure Portal. Create Azure Storage account or choose existing one to utilize for PowerApps. Create new container to store images from PowerApps like below. The new container name is "images" in this example.
f:id:waritohutsu:20200315071807p:plain

Pick up and save "Storage account name" and "Key1" into notepad to make "Connection" on PowerApps.
f:id:waritohutsu:20200315071903p:plain

Create new Connection for Azure Storage on PowerApps Studio for Web

Next, open PowerApps Studio for Web. Choose "Connection" from left menus. You need to create new connection for Azure Storage at first.
f:id:waritohutsu:20200315072200p:plain

You will find listed connections which someone has already created like below. Choose "New connection" to create newly your Azure Storage connection.
f:id:waritohutsu:20200315072327p:plain

Find "Azure Blob Storage" by using search box like below and click "+" button.
f:id:waritohutsu:20200315072450p:plain

Put "Storage account name" and "key" into inputboxes to complete this steps.
f:id:waritohutsu:20200315072558p:plain

This isn't mandatory, but I also recommend to change your connection name to find easily in later. Find you connection by sorting with "Modifed" like below.
f:id:waritohutsu:20200315072728p:plain

You can change your connection name like here.
f:id:waritohutsu:20200315072808p:plain
Now, you have completed to make new connection for Azure Blob Storage.

Create apps by using the Connection on PowerApps Studio for Web

Open PowerApps Studio for Web and choose to create Blank new app as first. Insert "Camera" control by choosing "Insert -> Media -> Camera" like below.
f:id:waritohutsu:20200315073135p:plain

Insert new button into your app. You will find two controls as "Camera1" and "Button1" like below.
f:id:waritohutsu:20200315073401p:plain

Next, add your connection for Azure Blob Storage. Choose an icon from left side and find your connection by following step an image below.
f:id:waritohutsu:20200315074037p:plain

Put formulas into "OnSelect" action on Button control by following an image below.
f:id:waritohutsu:20200315074500p:plain

Set( imagename, "driverface" & GUID() & ".png");
AzureBlobStorage.CreateFile("images", imagename, Camera1.Photo);

Run your PowerApps

Run your application and click button, so you can find images on Azure Portal like below.
f:id:waritohutsu:20200315074725p:plain