Speaking at TechDays Online, September 13th 2016

chefandazure

TechDays Online is an event hosted by Microsoft and the MVP community from the UK, containing bite-size technical sessions that offer a real-world perspective on how our technology is, and can be used by customers and partners.

I have a speaking slot on the “DevOps in practice” day, and will be talking to Marcus Robinson about my favourite topics: Infrastructure as Code, Azure Resource Manager templates and using Chef on Azure.

Monday, 12 September: Explore the world of Data Platform and BOTs
Tuesday, 13 September: DevOps in practice
Wednesday, 14 September: A day at the Office!
Thursday, 15 September: The inside track on Azure and UK Datacenter
Friday, 16 September: Find out more about Artificial Intelligence

The full agenda is here: https://www.eventbrite.co.uk/e/techdays-online-september-2016-led-by-mvps-tickets-27251539087

 

Announcing: Chef Extension for Visual Studio Code

At Microsoft’s Connect() conference today, Visual Studio Code now supports extensions and has been open-sourced!

Over the past couple of weeks I have been working hard behind the scenes to build a Chef Extension for Visual Studio Code to make cookbook development an awesome experience on Windows, OS X and Linux.

The extension currently supports:

Syntax/keyword highlighting –
Visual Studio Code will now recognise and colourise Chef DSL (including Chef Provisioning DSL). You can select from a number of different colour themes also available on the Marketplace.

Rubocop linting –
The quickest way to identify issues in your code is to use a linting tool like Rubocop to perform static code analysis as you go. The Chef extension is preconfigured to automatically run Rubocop against your whole repo every time you save.

Snippet support –
The extension includes autocomplete code snippets for all resources built into the Chef Client.

Here’s a demo of some of the new capabilities:vscode-chef-install

If you are working with Chef and you haven’t used Visual Studio Code before, I recommend you try it out. Downloads are available for Windows, OS X and Linux from https://code.visualstudio.com.  Once you have installed it you can download the Chef Extension directly from within the application’s Command Palette.

At Pendrica we are committed to open source, and making more awesome for the Chef community. We believe that Infrastructure developers using Chef should have the same rich feature set and extended user experience as our counterparts in application development. We hope you enjoy using the extension which is open-sourced on our GitHub page at https://github.com/pendrica/vscode-chef

Happy Cheffing!

Chef Provisioning with Azure Resource Manager preview

Those of you who are interested in provisioning in the Microsoft Azure world, I’ve released an experimental version of a Chef Provisioning driver for Azure Resource Manager (ARM) on GitHub.

The driver offers a very lightweight way of using Azure Resource Manager templates to provision complete application topologies, as well as using the Azure Chef VM Extensions to install and register Chef Client on each deployed node rather than bootstrapping externally (VMs created in Azure no longer necessarily have any internet access at boot time).

This driver should be seen as experimental/prototype, and there are some interesting limitations compared to how you might dream about a Chef Provisioning driver working… but I’m putting it out there for early feedback (PRs are very welcome too!)

The latest version is up on Rubygems and you can read further info at the README.md.

More to come in an upcoming blog post about some of the design decisions taken so far.

I’m writing a book all about using Chef with Microsoft Azure

awaiting-cover-image

This week, contracts were finalised and so I can announce that I’m writing a book all about provisioning with Microsoft Azure and Chef for Apress.

With a myriad of recent changes to navigate in Azure, Windows and Linux when it comes to automation and provisioning technology this is an opportunity for me to describe the current technical landscape and tools available in much more detail than my blog posts can – as well as providing some decent real world examples of why and where you want to use them.

If you want a primer for the book, the video from Ignite on Channel 9 with Michael Ducy from Chef and Kundana Palagiri from Microsoft is a good place to start. In fact, it was after attending this talk that I was approached by Apress to discuss writing the book.

The working title for the book is ‘Using Chef with Microsoft Azure’ and with any luck it should be published later this year.

Microsoft Azure and Chef taster day at BCS in London – 29th May 2015

chefandazureOur good friends at Chef are running a ‘Microsoft Azure/Chef taster day’ in London on 29th May 2015, where you will get to use Chef with Microsoft Azure, including tools such as Chef Provisioning.

This will be a very technical session and is a great opportunity to get your hands dirty using the latest integrations between Chef and Microsoft Azure.

Customers across the region are putting more and more of their IT services into Azure. To help you accelerate this trend CHEF are running a ‘Chef Taster Day’ on 29th May 2015 at the British Computer Society in London. The session is aimed at technical influencers in both our Azure partner community as well as Microsoft field technical teams.

As well as learning what Chef is and how it works you WILL BE CODING and producing something awesome on Azure which you can take out to your customers to show how they can move more of their business to our cloud.

Very limited (30) signups are available here: http://www.eventbrite.co.uk/e/microsoft-azure-chef-taster-day-tickets-16479612966

Chef Provisioning with Microsoft Azure – v0.3.2 released (Windows/WinRM support)

This post follows on from my earlier post Chef Provisioning with Microsoft Azure – part 1

It has been an incredible couple of months since I started contributing to the project with many new connections made and lots of interest in the tooling.

Yesterday at #ChefConf John Keiser released chef-provisioning 1.0 during his conference presentation in a fairly dramatic way:

As I have been given access to publish the Microsoft Azure driver to go along with it (thanks, John!) I have today released version 0.3.2 of chef-provisioning-azure to RubyGems.org.

This version is compatible with chef-provisioning 1.0 and chef-client 12.2.1 upwards.

Since my last blog post, we have enabled the functionality to:

  • create multiple VMs in a single cloud service
  • support launching and configuring Windows instances in Azure, using WinRM to deploy the chef-client immediately after boot
  • create VMs from user (captured) images as well as public ones
  • lower setup friction by supporting using the cross-platform azure-cli tools to configure defaults for authentication

To get the latest version installed into the Ruby installation that Chef uses:

chef gem install chef-provisioning-azure

Important note to chef-dk users: if you are still running chef-client 12.0.3 from the chef-dk, this is incompatible with the latest chef-provisioning-azure due to upstream dependency changes and you will need to update it to at least 12.2.1 using the instructions in https://github.com/fnichol/chefdk-update-app

Configuration changes

The driver now supports a wide variety of options to allow you to configure your credentials and subscription for access.  One feature I’d like to highlight is that access into your Azure subscription will ‘just work’ if you have previously used the azure-cli tools and imported your account settings.

To do that on your machine after installing the tools, all you need to do is:

azure account download
azure account import <downloaded filename>

We will then read in your subscription settings and certificate from your azureProfile.json file (it is stored in ~/.azure if you are interested) and if no subscription is explicitly set anywhere else (e.g. Chef client config or within the recipe itself) we’ll pick up your default subscription and use it to log you in.

Note: the previous way (versions 0.2.1 and below) of supplying logon information via ~/.azure/config has been deprecated.  You must now use one of the access methods specified in https://github.com/chef/chef-provisioning-azure/blob/master/docs/configuration.md

Windows/WinRM support

We now support the ability to bootstrap a Windows machine created in Azure using the WinRM protocol.  When the machine is provisioned, we ask Azure to enable WinRM over HTTPS support by default, adding the endpoint automatically to the cloud service.  Using this transport, the Chef Client is installed and it then registers the node with the specified Chef Server for your organisation.

Here’s an example recipe showing the options required to provision and bootstrap a Windows Server 2012 R2 image in your region:

 machine 'pendevxxapp15' do
  machine_options :bootstrap_options => {
    :vm_user => 'localadmin', #required if Windows
    :cloud_service_name => 'pendricacloud01', #required
    :storage_account_name => 'pendricastore01', #required
    :vm_size => 'Standard_D1', #optional
    :location => 'North Europe', #optional
    :tcp_endpoints => '3389:3389', #optional
    :winrm_transport => { #optional
      'https' => { #required (valid values: 'http', 'https')
        :disable_sspi => false, #optional, (default: false)
        :basic_auth_only => false, #optional, (default: false)
        :no_ssl_peer_verification => true #optional, (default: false)
      }
    }
  },
  :password => 'P2ssw0rd', #required
  :image_id => 'a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201502.01-en.us-127GB.vhd' #required
end

This recipe also demonstrates that you can specify custom TCP endpoints, which are a comma separated list in the format <publicport>:<privateport>.  Eventually we will support adding these endpoints to a load-balanced set.

Here’s a screenshot of the endpoints you end up with after using the above machine definition:

Virtual_machines_-_Windows_Azure

Here’s the sort of output you’ll get from running this recipe against a Windows machine:

Recipe: @recipe_files::/Users/SP/chef-repo.pendev/azure-provisioning-test.rb
 * machine[pendevxxapp15] action converge[2015-04-02T03:27:41+01:00] INFO: Processing machine[pendevxxapp15] action converge (@recipe_files::/Users/SP/chef-repo.pendev/azure-provisioning-test.rb line 24)
[2015-04-02T03:27:41+01:00] INFO: Reading azure profile file /Users/SP/.azure/azureProfile.json
[2015-04-02T03:27:41+01:00] INFO: Picked default subscription: Microsoft Partner Network (b6e7eee9-e4cd-41ae-8432-03ab624df016) from azure profile file /Users/SP/.azure/azureProfile.json

 - Creating pendevxxapp15 with image a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201502.01-en.us-127GB.vhd in North Europe...Creating deployment...
Cloud service pendricacloud01 already exists. Skipped...
Storage Account pendricastore01 already exists. Skipped...
Deployment in progress...
# # # # # # # # # succeeded (200)

 - Created pendevxxapp15 in North Europe...
[2015-04-02T03:28:38+01:00] INFO: Processing chef_node[pendevxxapp15] action create (basic_chef_client::block line 57)

 - create node pendevxxapp15 at https://chef.pendrica.com/organizations/pendev
 - add normal.tags = nil
 - add normal.chef_provisioning = {"reference"=>{"driver_url"=>"azure:b6e7eee9-e4cd-41ae-8432-03ab624df016", "driver_version"=>"0.3", "allocated_at"=>"2015-04-02 02:27:41 UTC", "host_node"=>"https://chef.pendrica.com/organizations/pendev/nodes/", "image_id"=>"a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201502.01-en.us-127GB.vhd", "location"=>"North Europe", "cloud_service"=>"pendricacloud01", "vm_name"=>"pendevxxapp15", "is_windows"=>true}}
 - Readying pendevxxapp15 in North Europe...
 - waiting for pendevxxapp15 to be ready ...
 - 0/120s...
 - 10/120s...
 - 20/120s...
 - 30/120s...
 - 40/120s...
 - 50/120s...
 - 60/120s...
 - pendevxxapp15 is now ready
 - Waiting for transport on pendevxxapp15 ...
 - 0/120s...
 - 10/120s...
 - 20/120s...
 - 30/120s...
 - Transport to pendevxxapp15 is now up![2015-04-02T03:32:51+01:00] INFO: Processing chef_node[pendevxxapp15] action create (basic_chef_client::block line 57)
[2015-04-02T03:32:56+01:00] INFO: Processing private_key[in_memory] action create (basic_chef_client::block line 78)

 - generate private key (2048 bits)
 - create directory C:\chef on pendevxxapp15
 - write file C:\chef\client.pem on pendevxxapp15

[2015-04-02T03:33:02+01:00] INFO: Processing chef_client[pendevxxapp15] action create (basic_chef_client::block line 131)

 - create client pendevxxapp15 at clients
 - add public_key = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQ
[...]
6YnnBwhgUYgWgFVxFZf0n8sE8\n4LcfQMHIHmk5CTKDVy4RynY7MYiwKl0XdNnrLFfAhItV4wNTm4bvE4gpNimYEX4f
\n3wIDAQAB\n-----END PUBLIC KEY-----\n"

[2015-04-02T03:33:04+01:00] INFO: Processing chef_node[pendevxxapp15] action create (basic_chef_client::block line 142)

 - Add pendevxxapp15 to client read ACLs
 - Add pendevxxapp15 to client update ACLs
 - write file C:\chef\client.rb on pendevxxapp15
 - run '(New-Object System.Net.WebClient).DownloadFile("https://www.chef.io/chef/install.msi", "$env:TEMP\install.msi")' on pendevxxapp15
 - run 'msiexec /qn /i "$env:TEMP\install.msi"' on pendevxxapp15[2015-04-02T03:35:35+01:00] INFO: Processing chef_node[pendevxxapp15] action create (basic_chef_client::block line 57)
[2015-04-02T03:35:36+01:00] INFO: Converging pendevxxapp15 because the resource was updated ...

 [pendevxxapp15] [2015-04-02T02:35:58+00:00] INFO: *** Chef 12.2.1 ***
 [2015-04-02T02:35:58+00:00] INFO: Chef-client pid: 2440
 [2015-04-02T02:36:57+00:00] INFO: Run List is []
 [2015-04-02T02:36:57+00:00] INFO: Run List expands to []
 [2015-04-02T02:36:57+00:00] INFO: Starting Chef Run for pendevxxapp15
 [2015-04-02T02:36:57+00:00] INFO: Running start handlers
 [2015-04-02T02:36:57+00:00] INFO: Start handlers complete.
 [2015-04-02T02:36:57+00:00] INFO: HTTP Request Returned 404 Not Found:
 [2015-04-02T02:36:57+00:00] INFO: Loading cookbooks []
 [2015-04-02T02:36:57+00:00] WARN: Node pendevxxapp15 has an empty run list.
 [2015-04-02T02:36:58+00:00] INFO: Chef Run complete in 0.970965 seconds
 [2015-04-02T02:36:58+00:00] INFO: Running report handlers
 [2015-04-02T02:36:58+00:00] INFO: Report handlers complete
 - run 'chef-client -l info' on pendevxxapp15
[2015-04-02T03:37:01+01:00] INFO: Chef Run complete in 564.373827 seconds
[2015-04-02T03:37:01+01:00] INFO: Unable to access cache at /var/chef. Switching cache to /Users/SP/.chef

Running handlers:
[2015-04-02T03:37:01+01:00] INFO: Running report handlers
Running handlers complete
[2015-04-02T03:37:01+01:00] INFO: Report handlers complete
Chef Client finished, 1/1 resources updated in 568.259339 seconds

You can see that the VM ran Chef Client and converged.  At this point you are then able to manage the node in the usual way.  For example the command

knife node show pendevxxapp15

Will return basic information about the node:

Node Name: pendevxxapp15
Environment: _default
FQDN: pendevxxapp15
IP: 100.78.32.54
Run List:
Roles:
Recipes:
Platform: windows 6.3.9600
Tags:

User (captured) image support

If you have captured an image on Azure with base software in, and now need to create machines from it, you can!

Simply specify the VHD name for your captured in your recipe and the driver will work out which type of image it is.  Note that VHD images need to be stored in the same storage account as your new machines.

Here’s an example recipe:

machine 'mynewmachine' do
  machine_options :bootstrap_options => {
  :cloud_service_name => 'mycloudservice',
  :storage_account_name => 'mystorageaccount',
  :vm_size => 'Small',
  :location => 'North Europe',
  :tcp_endpoints => '80:8081,443:9443'
  },
  :password => 'P2ssw0rd',
  :image_id => 'capturedvhdimage-20150223-402981'
end

Full range of VM role sizes now available

One other change made in this release was to point to our own version of the Azure SDK for Ruby.  Quite simply this is because it hasn’t been kept up to date with the latest VM role sizes.  In this release we fixed that so at the time of writing the possible :vm_size values for a VM role are:

 Basic_A0 Basic_A1 Basic_A2 Basic_A3 Basic_A4
 ExtraSmall Small Medium Large ExtraLarge A5 A6 A7 A8 A9 A10 A11
 Standard_D1 Standard_D2 Standard_D3 Standard_D4 
 Standard_D11 Standard_D12 Standard_D13 Standard_D14
 Standard_DS1 Standard_DS2 Standard_DS3 Standard_DS4
 Standard_DS11 Standard_DS12 Standard_DS13 Standard_DS14
 Standard_G1 Standard_G2 Standard_G3 Standard_G4 Standard_G5

It goes without saying that these machine types need to be available in your geography to be available for your subscription.

Finally

Finally a couple of plugs — if you didn’t see it and are interested in the basics of Chef Provisioning on Azure, you can watch the venerable John Keiser, lead developer at Chef demonstrate the basic features on Channel 9 with David Tesar from Microsoft.

Issues are raised and tracked in the GitHub repository at https://github.com/chef/chef-provisioning-azure and we’d love to hear your feedback and suggestions there.

Keep an eye on this blog and on twitter @stuartpreston for more announcements coming soon!

Enabling the Chef VM Extension on a Microsoft Azure VM using Powershell

As I mentioned in an earlier blog post, there are many ways to install and configure (“bootstrap”) a Chef client when using Microsoft Azure.

One of these ways is by enabling the AzureVMChefExtension. One particular benefit of this is that the Chef client config is dynamically generated and injected into the machine through the framework, negating the requirement to have opened up an SSH or WinRM/s endpoint pointing at your server before you have configured and secured it. This is necessary in the Azure world because (unlike AWS) the Cloud Service providing the external IP through which you access the machine externally may have multiple VM instances behind it (e.g. they could be participants in a Load Balanced set, or simply running on different ports), making a robust external bootstrap process difficult.

Anyway, here’s a gist demonstrating how to create a VM with the extension enabled from a Powershell script (note: you must have the Azure Powershell Cmdlets installed):

The script should be saved to the root of your chef-repo and paths and variables altered accordingly. After executing the script and waiting for the server to be provisioned, you’ll see the extension installed in the Management WebUI:

chef-service-enabled

Assuming your Chef server is set up for SSL correctly, the node will self-register with the specified Organization on your Chef server (using the specified validation key) and it is now ready for management using Chef:

chef-server-registered

Hope this helps someone.

Chef Provisioning with Microsoft Azure – part 1

Introduction

Chef Provisioning promises to “idempotently create and converge machines, images, load balancers and other infrastructure, no matter where they are” – so we are talking about the description and provisioning of complete environments in which to run applications and workload rather than individual nodes of an infrastructure.

The idea of combining continuous delivery techniques with Azure is pretty exciting – as new requirements come up, you modify your environment description (for example to add or remove an endpoint from an existing Cloud Service, or add an ACL) and push out this configuration via a Chef Client.

This blog post will help you get the basics of Chef Provisioning up and running with the Microsoft Azure IaaS platform, using both Windows and Mac client platforms to do the provisioning.

Background

Personally I like to separate my concerns of Infrastructure/Environment provisioning from those of Application Release Management, as much as I separate Build/CI/Application Packaging from Application Release Management – keeping the separation improves our ability to use exactly the same processes to create, test, manage, deploy and monitor – by the time we get to Production the same code has been exercised many times and there are fewer surprises along the way.

The approach provides flexibility for individual pieces to be swapped out as newer and better tools come along.  It’s a shame to see so many Dev shops going against this though (e.g. “oh we deploy to our dev and test environment from <insert build/CI tool here>, but use <insert release management tool here> to deploy to Staging and Production”).  But I digress…

As I’ve been investigating provisioning IaaS on Microsoft Azure recently, it definitely makes sense to be able to provision the higher level resources such as Cloud Services, Endpoints and Virtual Networks using a common language and tooling that works cross-platform.  Enter chef-provisioning (formerly chef-metal) and specifically chef-provisioning-azure, a driver for Azure that works with the constructs and resources provided by Chef Provisioning.

Now what is immediately clear is that chef-provisioning-azure is a long way behind it’s sibling drivers (such as chef-provisioning-aws) and this is made clear by the Readme.md:

health-warning

But having made the effort to get things working I thought I would share the important information you need to get going.

Prerequisites

The steps below are for either Windows or Mac based systems, I have tested on both platforms.

ChefDK

First of all, if you don’t have it already – we need the latest “chef-dk” on our client machine – this provides a local chef-client, linting, testing tools and importantly a version of Ruby that is required to install Chef Provisioning and any required drivers.  I recommend a version of 0.4.0 or above.

It can be downloaded from https://downloads.chef.io/chef-dk and we can verify the version using the command:

chef -v
Chef Development Kit Version: 0.4.0

Resolve path to Gem [Windows only]

After installation, we need to make sure that the path contains the path to Gem, the Ruby package manager.  On my Windows system I had to manually add the following to the end of my PATH variable and restart my Command Prompt:

c:\opscode\chefdk\embedded\bin

On the Mac the path is:

 /opt/chefdk/embedded/bin

Installing chef-provisioning-azure gem from the repository

As the ChefDK 0.4.0 installed Chef Provisioning 0.18 for us, all we need is the Gem for chef-provisioning-azure to be installed.  At the time of writing this is v0.21 and we will need to make some local changes to make it work:

chef gem install chef-provisioning-azure

You should see a message similar to:

Successfully installed chef-provisioning-azure-0.21
1 gem installed

Microsoft Azure Management Certificate installation

To allow authentication of requests to Azure’s Service Management API, we need an x509 v3 certificate in either a .pfx or a .pem format.

To retrieve the .PFX file required, follow the steps in my previous blog post:

http://stuartpreston.net/2015/02/retrieving-microsoft-azure-management-certificates-for-use-in-cross-platform-automationprovisioning-tools/

The resulting .pfx file needs to be placed in the following locations:

Mac:

~/.azure/mgmtcert.pfx

Windows

%USERPROFILE%\.azure\mgmtcert.pfx

Microsoft Azure configuration file

Azure configuration for Chef Provisioning is expected to be found in a file called ~/.azure/config or %USERPROFILE%\.azure\config depending on your platform.  Eventually this config will support multiple profiles/subscriptions.

The file should look something like the following:

Mac:

[default]
management_certificate = "/users/StuartPreston/.azure/mgmtcert.pfx"
subscription_id = "b6e7eee9-12ab-12ab-12ab-03ab624df016"

Windows:

[default]
management_certificate = "c:\users\Stuart\mgmtcert.pfx"
subscription_id = "b6e7eee9-12ab-12ab-12ab-03ab624df016"

(the subscription id can be found within the downloaded .publishsettings file against your subscription name)

We will discuss in a later post how to configure this for execution via a remote Chef node that runs under a different user account.

Ok we’re ready to see things in action now…

Provisioning a Azure Storage Account/Cloud Service/Virtual Machine

For this basic test, we will use a very basic example just to verify we can create and destroy machines within our target correct subscription within Azure.

In a new file, azure-provisioning-test.rb add the following text:

require 'chef/provisioning/azure_driver'
with_driver 'azure'
machine_options = {
 :bootstrap_options => {
 :cloud_service_name => 'pendricachefcemo01',
 :storage_account_name => 'pendricachefdemo01',
 :vm_size => "Small",
 :location => 'West US'
 },
 # Until SSH keys are supported (soon)
 :password => "P2ssw0rd"
}

machine 'penchefdemo01' do
 machine_options machine_options
end

As we have not specified an image name, the above will create a Small size Linux VM using a default Ubuntu image in the West US region.  You’ll need to change the cloud_service_name, storage_account_name and machine name to something unique, remembering the Microsoft Azure rules on naming (it is fair to say the Azure Service Management API is less than forgiving if you get this wrong, there is little to no remediation in the chef-provisioning driver just yet and this can leave you with a Cloud Service with no Storage or VM for example).

Once the file is updated we are going to run it through our local chef-client.  At the command line type:

chef-client --local ./azure-provisioning-test.rb

If you got the config right, you should see output similar to the following, with the Storage account, Cloud service and VM being provisioned:spvm2015_cloudapp_net 3

If you got a more serious error, you can re-run the above in a more verbose debug mode:

chef-client --local ./azure-provisioning-test.rb -l debug

If all went well, you should see the machine being provisioned in the Azure management portal:

spvm2015_cloudapp_net 2

The chef-client run will probably complete but with errors, this is to be expected really, not least because we didn’t configure the bootstrap process at all – we just wanted to demonstrate some interaction with Azure.

Now we want to destroy the VM we created, remember we’re being declarative about this so we need to add the destroy action to our machine by adding a single line to our recipe:

require 'chef/provisioning/azure_driver'
with_driver 'azure'
machine_options = {
 :bootstrap_options => {
 :cloud_service_name => 'pendricachefcemo01',
 :storage_account_name => 'pendricachefdemo01',
 :vm_size => "Small",
 :location => 'West US'
 },
 # Until SSH keys are supported (soon)
 :password => "P2ssw0rd"
}

machine 'penchefdemo01' do
 machine_options machine_options
 action :destroy
end

Then re-run the command:

chef-client --local ./azure-provisioning-test.rb

What should happen now is something similar to the following:

spvm2015_cloudapp_net 6Don’t forget to verify that the machine and cloud service has actually been deleted in the Azure Management portal.  I noted that the Storage account was left behind, this may need cleaning up manually if it is a concern.

And naturally this works cross-platform too, just to prove it here is the same process (create and destroy) running on my Mac:

Create:

stuartpreston_—_ruby_—_81×47

Destroy:

stuartpreston_—_bash_—_81×47

Conclusion

It is probably fair to say that the chef-provisioning-azure driver is still in very early stages of development.

The functionality at the moment lets us create, test for compliance, modify and destroy Storage Accounts, Cloud Services and VMs – of course in the real world of Azure IaaS we have deployment slots, endpoint monitoring, affinity groups, storage account naming rules, custom metrics, regional differences (not all services are available in all Azure datacentres) and many more configuration aspects to deal with.

The responsibility is still on the user to have knowledge of the Azure rules and conventions – for example: don’t try and create a Virtual Machine using an existing Storage account that is not in the same region, this will fail (horribly) in the same manner in which it would if you tried to do the same using the Azure Management Portal… additionally you have to remember the (often inconsistent) rules of naming objects in Azure (e.g. a Storage account cannot have hyphens in it)

In summary, there are many limitations at the moment and it will be a while before this becomes fully featured and production-worthy.  But I am very positive about the future and direction of Chef Provisioning as a whole.

New possibilities have been opened up and I look forward to contributing and having all my pull requests accepted!

I will be blogging lots more about fixes being made to the driver – as well as updating this blog post as new updates come to light, so watch this space!

Further reading


Edit: This post was updated 19th March 2015 to incorporate feedback and represent the current published state of the project.

Bootstrapping Chef clients on Microsoft Azure

I’ve been working with Chef on Microsoft Azure (and more recently, Chef Provisioning – the subject of a much longer blog post which I am still busily writing…watch this space!)

I came up with the below list of existing options for “bootstrapping” a Chef client created in Azure and thought I would share them here.  The first two are not Azure-specific and hence probably the most popular, the rest are definitely only applicable if you are actually using Microsoft Azure.

In the Chef world, nodes are ‘bootstrapped’ and the clients communicate with the Chef Server which can either be hosted in your environment or there is a public hosted version  – and until now, many admins would bootstrap either by hand, by a script or via an image.  In the case of Microsoft Azure, you have a few options to get a server “bootstrapped” on an individual basis:

  1. If the machine is already created and is a Linux machine and you have access to port TCP/22 either directly via a Virtual Network or a Cloud Service endpoint – see https://docs.chef.io/knife_bootstrap.html
    knife bootstrap
  2. If the machine is already created as a Windows Server 2012 or higher machine – and you have the knife-windows plugin installed on your local machine – and winrm is configured to use the HTTP transport, and the endpoint configuration for the cloud service opens the required port (it’s TCP/5985 by the way and not added to a cloud service by default) or you have lots of time on your hands to work through the SSL validation requirements and a way of getting the certificates from the server – see http://docs.chef.io/plugin_knife_windows.html
    knife bootstrap windows winrm
  3. If you have downloaded your publish settings file and haven’t created a virtual machine yet – this tool offers a simple way to get up and running – see http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-automation-with-chef/ :
    knife azure
  4. You can search for a Linux VM Image in the gallery with the client already in it – though you still have to get the required keys into the server. NB: only appears to be available in the new portal.portal-chef
  5. You can add the Chef Resource Extension for Azure Virtual Machines on creation of a Windows 2012 or higher virtual machine by hand – this is nice because you can store the client configuration and required keys in blob storage and have the VM retrieve them – see http://blogs.technet.com/b/dcaro/archive/2014/04/22/hands-free-configuration-of-your-vm-in-microsoft-azure.aspx :042214_1035_Handsfreeco6
  6. Finally you can install Chef Resource Extension for Azure Virtual Machines via Powershell – this requires a recent version of the Azure Powershell Cmdlets installed – see https://msdn.microsoft.com/en-us/library/azure/dn874067.aspx and I will also blog my experience with this approach when I can. Update (13th February 2015) here’s a blog post I’ve created that shows how to accomplish this: http://stuartpreston.net/2015/02/enabling-the-chef-vm-extension-on-a-microsoft-azure-vm-using-powershell/