Image

DarylsCorner

Cloud Computing in Today's World

Monthly Archives: February 2016

Overview of Azure Service Fabric

/ / 1 Comment

Today, in part 2 of this series, I wanted to continue down the path of exploring Azure Service Fabric by presenting an overview of Azure Service Fabric. In part 1 of this series we talked about why you would want to use something like Service Fabric. That led us to a primer discussion around microservices which just so happen to run on the Service Fabric platform. If you haven’t done so already, you may want to read part 1 of this series first as it will help you build a better understanding.

My Journey to Service Fabric

Recently I was working on a project where the customer had an application that was hosted in Azure. They were utilizing the Cloud Services platform but soon came to terms that it may not be the best option for their application. With Cloud Services there are some limitations around being able to scale an application, primarily dependent upon how the application was architected. They soon realized that they needed a solution where their application workload would be scalable, reliable, and manageable. They had already begun to start working on a microservices approach for the application so it was only fitting that they explore Service Fabric. Hence, why I decided to write this blog series. I figured I could help others down the Service Fabric road to becoming DevOps Unicorns. So let’s see what Service fabric is all about.

What is Service Fabric

Service Fabric in its most simplest form is a platform for running microservices. It can’t be that simple right? Well, it is quite simplistic but there is much more to it than that. Service Fabric is a distributed systems platform that simplifies the packaging, deployment, and manageability of scalable and reliable microservices. This platform significantly changes how developers and administrators approach the deployment and management of mission-critical workloads. No longer do they have to deal with building and managing complex infrastructures. Instead they can focus their efforts on their application, increasing productivity and customer satisfaction.

Service Fabric itself is a shared pool of servers known as the Service Fabric Cluster. The cluster is made up of multiple nodes that host distributed microservices, those being stateful or stateless. By using microservices we have the ability to scale services independently of each other which allows developers to push out updates and fixes more frequently, not to mention reducing the chances of a broken service affecting the entire application. In addition, Service Fabric provides application management capabilities for provisioning, deploying, monitoring, upgrading, patching, and deleting of services. From a natural progression of application hosting we have gone from a physical server, to a virtual machine, to a container, to microservices. So whats next? Nanoservices? There is a concept of an application being too fine-grained and requiring more effort to stitch the services together but we won’t worry about that now. The image below shows a good representation of Service Fabric with many microservices deployed on top. Not only can Service Fabric be deployed in Azure, but it can be deployed in other clouds as well as on-premises.

 

Traditionally stateless applications required a database of some sort to maintain the state along with caches and queues to address latency. One of the problems with this is that it required more components to manage that didn’t scale really well together. In addition, each additional connection added to the latency. By using stateful microservices we remove the need for those additional components and can maintain high-availability and low-latency by keeping the application code close to the data.

Within Service Fabric there is support for application lifecycle management (ALM) from development, through deployment, during management, and decommissioning. By utilizing packaged microservices, multiple instances can be deployed and upgraded independently. Rolling upgrades can be performed to ensure the application is always available. In the event an upgrade fails, automatic rollback will kick in.

Additional Capabilities

  • self-healing applications
  • run Service Fabric on your laptop as a dev environment; same code runs in Azure
  • services can be built using framework of your choice
  • applications deploy in seconds
  • write once, deploy anywhere
  • deploy on Windows Server or Linux (coming soon)
  • deploy hundreds or thousands of applications per machine
  • manage using .NET APIs, PowerShell, or REST
  • monitor and diagnose the health state
  • scale up or down the cluster, capable of scaling to thousands of machines
  • redistribute and optimize load after failure

How Service Fabric Works

Briefly let’s discuss how Service Fabric works. Refer to the Azure Service Fabric documentation for a deeper dive.

In Service Fabric you have a cluster of nodes running a Windows service called FabricHost.exe that auto-starts upon boot. This service starts Fabric.exe and FabricGateway.exe which make up the node. Note: When running on your laptop as a dev environment you essentially just run multiple instances of these executables which show up as multiple nodes. An application package that references the service packages for each service type are copied to the image store. Once the package has been copied you can create an instance of the application within the cluster by specifying the type. The application type instance is then assigned a URI name of “fabric:/MyNamedApp“. Within the cluster you can create multiple named applications, each of which are managed and versioned independently. Named services instances are created under its named application which looks like “fabric:/MyNamedApp/MyNamedService“. During the named service creation you specify a partition scheme which spreads across cluster nodes allowing for scale. Within a partition stateless named services have instances while stateful named services have replicas which are kept in sync. Should a replica fail, Service Fabric builds a new replica from existing replicas. In addition, during named service creation, code, data and configuration packages are copied to the node(s) running the service where they can be used. You can create two types of service:

  • Stateless: use when persistent state is not required or using external storage
  • Stateful: use when persistent state is required. Uses Reliable Collections or Reliable Actors programming models.

Service Fabric Architecture

The diagram below shows the major subsystems of Service Fabric with a brief description of each.

  • Transport Subsystem – provides secure communication between nodes
  • Federation Subsystem – creates cluster of nodes into single entity for detecting failures, perform leader election, and provide consistent routing
  • Management Subsystem – manages lifecycle of applications and services
  • Testability Subsystem – help devs test services through simulated faults before and after deploying applications and services to production
  • Communication Subsystem – resolve service locations
  • Reliabilty Subsystem – responsible for reliability through replication, resource management and failover
  • Hosting and Activation – manages lifecycle of an application on a single node
  • Application Model – enables tooling
  • Native and Managed APIs – exposed to devs

In Summary

Service Fabric is being termed the next-gen platform for building and managing cloud-scale applications. If you are interested in seeing how this works or even deploying Service Fabric, stay tuned for the next post in this series where we will Deploy Service Fabric.

Exploring Azure Service Fabric

/ / Leave a comment

This is the first in a series of posts about Exploring Azure Service Fabric. My goal here is to give you an idea of why you may want to use Service Fabric, provide an overview of the service itself, and demo a few deployment options. Below is a list of posts that I plan to include in this series so stay tuned if you find this of any interest..

In this first part of the series I wanted to talk about some of the driving factors for using something like Service Fabric. If you are reading this you probably are already familiar with the term microservices. If not, Wikipedia defines the term like so. “Microservices is a software architecture style in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a modular approach to system-building.”

Over the last few years microservices have emerged to replace the traditional tiered architectures and monolithic applications that are tightly coupled. So why is this? Think back to the days when you needed to update your application with a new feature, or worse, you hit the boundaries of your application and needed to scale up or out. It was no easy task and it caused updates and fixes to build up until a new version of the application was released. As a business, if you weren’t able to evolve your application to better suit your end users, then it slowly faded away as people stopped using it. With today’s technologies, when we talk about building applications, you need to consider the scope. Most conversations lead to the cloud in some form or fashion so we need to ensure that the applications are designed for scale and capacity. With scale and capacity being unpredictable, the cloud becomes an ideal platform for running our applications. If you need to scale it becomes much easier, provided that your application has been designed as a collection of fine-grained autonomous services that have their own lifecycles and collaborate together. This allows you to get feature updates and enhancements to your customers at a much faster pace all while continuing to improve your application based on customer feedback. This is what is known as the lean startup model wherein you start with an idea, build (code), measure (data), and learn (pivot if needed). This in turn creates more reliable releases. Does any of this sound somewhat like the concept of DevOps? Sure it does, after all our goal is to accelerate the flow of work through dev, test, and IT Ops in a way that speeds application delivery and increases efficiencies.

With that said, it isn’t as simple as flipping a switch and you go from a monolithic app design to a microservices app design. In some cases you may need to start with a monolithic approach and slowly move towards a microservices design. This sort of approach would allow you to slowly decompose the app starting with lower tier services that need to be more scalable, all while moving towards a microservices approach. Additional benefits of microservices include language-agnostic, interaction with other microservices, resilient to failures, and self-healing services. The end goal of a microservices approach is to compose your application into smaller, autonomous services running in containers across a cluster of machines.  This allows smaller teams to focus on a service that is independently tested, deployed, scaled, and upgraded. Can you start to see how all of this fits together, the DevOps culture, the lean startup model, and microservices.

Microsoft was used to delivering applications in a monlithic form but soon realized they needed a way to deploy large scale services like Azure SQL databases, DocumentDB, along with other core services, that allowed for independent team development, was scalable and reliable, with low latency. This is where Service Fabric comes in. Essentially it is a platform for running microservices. We will get into more of its capabilities in a part 2 of this series.

An important concept that I want to cover, before we move on, is without a framework of principles and practices to guide your design, it will be quite difficult to deliver and maintain a microservices environment. You and your teams will want to review these and put together a framework covering these principles. Doing so will help you achieve success.

  • Strategic Goals: high-level goals defining where the company is going so the technology is aligned to best suit the customer (ie. expand into European and Asian markets)
  • Principles: rules defined that align with what you are doing from a strategic goal. Keep it small – no more than 10 so they can be remembered (ie. allow for portability)
  • Practices: how to ensure the principles are carried out. Detailed guidance for performing tasks (ie coding and design guidelines)

Note: For smaller teams you may end up combining principles and practices.

Being able to articulate this in the form of a chart  to summarize these concepts will be most useful and readable. This could be something as simple as the below diagram.

For additional information on this topic check out the book  “Building Microservices” by Sam Newman.

Stay tuned for the next post in this series where we will dig deeper into an overview of Service Fabric and its features.

Red Hat Enterprise Linux Images in the Azure Marketplace

/ / Leave a comment

Today Microsoft announced the availability of Red Hat Enterprise Linux images in the Azure Marketplace. You can now deploy RHEL 6.7 and 7.2 images in all regions except China and US Government. This is a big move for Microsoft and it shows their commitment to the open source ecosystem. Microsoft will offer fully integrated support with Microsoft and Red Hat support engineers sitting along side one another. Lets see what the deployment experience from the Azure Portal looks like.

Deploy from Azure Portal

You will find the image by clicking on the add new button in the portal.

A new blade will open, type Red Hat to search the Marketplace. Here you will see two version to choose from. Select your desired version.

Now you can choose your image.

A new blade will open, keep the default deployment model set for Resource Manager and select Create.

From the Create Virtual Machine blade, first fill out the Basic settings of VM name, User name, Authentication type, Password, Subscription, Resource group, Location and click OK.

Next we need to choose a Size for the VM.

Next we need to configure the optional features. You can chose to keep the defaults if you are just testing but you would want to change these to something more meaningful beyond that. Here you can choose the Disk type (standard or premium), Storage account, Virtual network, Subnet, Public IP address (dynamic or static), Network security group, Diagnostics (enabled by default), Diagnostics storage account (if different from blob storage account), and whether you want to use an Availabilty set and click OK.

Review the Summary and click OK to start the deployment. One thing I noticed is there wasn’t an option to not pin the VM to the dashboard. I imagine that this will get fixed soon. From the dashboard you can just right-click on the VM and choose to unpin it. Allow a few minutes for the deployment to succeed.

You can now see a summary of the resources deployed in your Resource group.

If you click on the VM resource and on the Settings blade you will see a new item called Red Hat Customer Portal which will give you direct access to the portal.

To connect to the VM you can use Putty. Enter the Public IP address of the VM using SSH on port 22  and click Open.

Login with the User name and Password and you are all set.

Here we can see the release info.

Now prepare yourself for whats coming next. Viola! Check out the Hypervisor vendor.

Deploying a Red Hat Enterprise Linux image from the Azure Marketplace is pretty simple so go out there and deploy and have fun.

 

 

Copy Status of an Azure Storage Blob

/ / Leave a comment

Have you ever had the need to check the copy status of an Azure storage blob? Recently I was working on a project where I had a requirement to use a custom image for an Azure deployment utilizing an ARM template. One of the caveats of using a custom image is that it needs to be staged on the target storage group prior to deployment. This is not that easy since the target storage group probably doesn’t exist just yet. This is not a very elegant way of having to do something and hopefully Microsoft will come up with a better way to handle this soon.

With that said, one of the issues I faced was knowing when the copy process had completed so I could move forward with the template deployment. Depending on what cluster or region your storage accounts reside in, will determine how long it may take to perform the copy. Unfortunately since there isn’t a way to perform the copy using an ARM template or even in the Portal for that matter, I wonder if this can be accomplished via PowerShell. Of course it can! PowerShell can do anything, well almost. However, it does help when there are “Help Topics” for ARM cmdlets, hint, hint. (Microsoft please fix this!)

Ok, so let’s check this out. It is assumed you already have a resource group (xyz-rg) with a storage account containing your custom image. In my case I have a storage account called ‘xyzstor1’ with a container called ‘myimages’ that contains a sysprepped, generalized image, all within its own resource group. This would be considered as your source storage account.

In addition, I have also deployed a new resource group for the target deployment that contains a storage account and container named accordingly. Let’s call our target storage account ‘defstor1′ and the container “myimages’. Now we need to copy our source image to the target storage container.

First, lets define our source storage account, container and context.

Next, lets define the target storage account, container and context. Note: You will need to define the $newRGName variable.

Here we  want to define the $blob variable with the name of your ‘image’ and reference the blob in the source container.

Now we kick off the blob copy from source to target.

Now that the copy is in progress, we need to know when this is complete so we can continue. If we just run the following command you can see that PowerShell returns a result immediately making you think the copy has completed. Wow, that is fast. Well, not exactly. You can see that the Status is actually Pending. If you tried to run your ARM template you would get an error stating that the blob could not be found.

But, if we include the -WaitForComplete parameter we can actually see the status of the copy and we’ll be able to know when it has completed as seen in the screenshot below.

You may wonder why the copy is taking a while. In my case I am actually copying across regions from ‘West US’ to ‘South Central US’. Within the same region you should see faster results. After about 15 mins I can now see that the copy has finished and I can continue with my ARM template deployment using the staged custom image.

Once the ARM deployment has finished you no longer need to keep the staged custom image and you can delete it. Make sure you are deleting the copy and not your OS vhd. This should be easily distinguishable as the OS vhd should be in a container named ‘vhds’, if you have followed the standard container naming format.

I hope that this has been informative and that it may help you verify the copy status of an Azure storage blob.

 

BONUS: Curious to know which storage stamp your storage account resides on? Just ping the storage account, although don’t expect a reply. In the case of ‘defstor1’, it is sn3prdstr06a.

 

 

TwitterRSS