Introducing Polymelia Deploy

During the last month I have created different deployment tools, as a proof of concepts. The tools have change from push deploy into pull deploy, from an own XML workflow and environment definition into using Microsoft Workflow. Finally I decided to introduce to you the Polymelia Deploy tool. The goal of the tool is to make it open source. The code I have is still in a proof of concept level, and need some more work until it will be available.

Polymelia uses agents installed on servers. By using pull deployment, no one can communicate directly to an agent. This make it much easier for to install agents on servers and no ports need to be opened. Each agents have a role. For example a role as a “Web Server”, or “Application Server”. When an agent is running it will ask a Controller for tasks to be executed.

 

Because agents has roles and Polymelia uses Pull deploy, we can now add a new server, put an agent on the server, specify the role, for example “Web Server”. When the server is up and running the agent can ask a controller for tasks. The latest succeeded tasks will be retrieved and executed. That makes it easy to just add a new server to a load balancing environment and get it auto configured and installed when it’s up and running. No need to do a push deploy, or do changes to the deploy script.

 

In a near future the agents will be able to be scheduled, when and how often it should ask for a task. The agents will also use SingalR, and by using SingalR, a controller can know when a new agent is added to the environment, and by suing Polymelia Deploy Client, we are able to approve that agent before it can ask for a tasks. Some ideas on the list to do, is to be able to specify an IP range for auto allow new agents without needing to approve them.

Polymelia have as the moment just a few activates (but will get more, maybe you will help me create them ;)), one activity is a NuGet Installation activity, it has a similar solution as Octopus Deploy. The activity will get binaries from an Artifact Repository using NuGet server.

 

 

The packages can have configuration files that can be transformed, variables that can replace connection string, appsettings keys and WCF endpoints, but will in a near future replace all kind of keys and values in the configuration file using markers in the config file:

<appSettings>
<add key=”$VariableName$” Value=”$variableName2″/>
</appSettings>

 

The NuGet Installation activity will also search for PowerShell scripts in the package, pass variables and execute the script. It will make it possible to use PowerShell to configure and install packages on a server. Because Polymelia is based on Microsoft Workflow, it’s possible to use pre-defined activities that will reduce the use of PowerShell, like creating a MSMQ, Install a Service, Create an app Pool, Run PowerShell script and Start a Virtual Machine etc.

Polymelia Deploy Client

 

Polymelia Deploy Client is the tool to create deployable workflows, and is used to perform the deployment of a specific version.

When Polymelia Deploy Client is loaded we can create or select a project:

 

When a project is created or loaded, we are able to add environments:

 

When the environment(s) are added we can start creating our deploy tasks. The following illustrate how we can tell the Controller to start a Virtual Machine, the Virtual Machine has an agent installed with the role “Web Server”. When the Virtual Machine is started a parallel activity is started and will execute two “Deploy to Agent” activities. One to the role “Web Server” and one for the role “Database”. The tasks added into the “Deploy to Agent” are the tasks that the Controller will add to a queue. The “Web Server” role will read from the queue to execute the tasks added for that role. The “Web Server” will get two packages from a NuGet server and install them on the server, this is done in parallel.

 

When hitting the DEPLOY button, we need to specify the version we are going to deploy, and the deploy workflow will then be passed to the Controller for execution. When the agents is starting to install tasks, they reports back to the Controller and the client can read from the reports.

This project is still under development. If you are interested to contribute to this project, please let me know. The reason I make this tool Open Source, is to get help of the community to build it, and also give an open source option for the community to use.

If you want to know when I post a blog post, please feel free to follow me on twitter: @fredrikn

Read More

Bra junit- & Mockito-favoriter och templates för eclipse

Favoriter junit.framework.Assert.* org.mockito.Mockito.* Templates tt (skriv tt ctrl+space där du vill ha en testmetod): @${testType:newType(org.junit.Test)} public void ${testName}() throws Exception { ${staticImport:importStatic('org.junit.Assert.*', 'org.mockito.Matchers.*', 'org.mockito.Mockito.*')}${import:import('org.junit.Test', 'org.junit.Before', 'org.mockito.Mock')}${cursor} } mo (skriv mo ctrl+space på raden innan klassdeklarationen för att få stöd för automocking): ${staticImport:importStatic('org.junit.Assert.*')} import static org.mockito.Matchers.*; import static org.mockito.Mockito.*; ${:import(org.junit.Before, org.junit.Test, org.mockito.Mock, org.mockito.runners.MockitoJUnitRunner, org.junit.runner.RunWith)} @RunWith(MockitoJUnitRunner.class) Tack […]

Read More

Deployment tool

During the last twelve months I have spent a lot of time on Continuous Delivery and a deep dive into Team Foundation Server 2012. A commit stage is set, we use TFS build to build, and NuGet as an Artifact Repository. Now my goal is to bring the deployment to the team, let them also be part of the deployment process, maintain and create deployment scripts etc. I have looked around among some deployment tools like Octopus and InRelease. InRelease looks promising. Instead of using one of the great tools, I decided to create my own. Why you may ask? The reason is that I want to be able to do any modifications and bring the code to the team, and to be honest, I needed something to do on my spare time 😉

In this blog post I will describe my tool.

Pull or Push deployment

 

My first tool supported only Push deployment. It uses my own XML definition do specify environments and deployment steps. My deployment controller at that time read the XML and downloaded the packages to deploy from a NuGet repository and pushed them to deployment agents (Installed on every machine). The deployment agent role was to unzip the package and do configuration files transformations and to run PowerShell scripts. By using push deployment, I have more control over how the deployment went. The main problem was that I needed to make sure the Deployment controller had access to the server where the agents was installed, and also that a specific network port was open in the firewall. Another problem is when a new web server will be added (for example to a load balancing), changes need to be made to the XML file specifying the deployment environments, also a new push deployment need to be started to configure the server.

I decided to move over to a more pull deployment solution. I removed my own XML definition file and decided to use Microsoft Workflow instead. The Deployment controller uses OWN and Katana instead of WCF (my first push tool uses WCF). The deployment controller has the responsibility to execute a deployment workflow. I added a workflow activity that can be used to perform action on specific servers. When a deployment is started the deployment control added the task to a queue. The deployment agent checks the queue for a task. If a task was added to the queue for the specific server, the agent started to execute the task, and report back the progress to the deployment controller. The deployment agent downloads the packages it needs, it also transforms configuration files and execute PowerShell scripts if they was part of the package to be installed. Nothing form the queue will be removed, the deployment agent hold the responsibility to know which task it has already “executed”. By doing so I can simply install an agent to a new server, specify that it has the role as a “webserver”, and when it’s started, it can check the queue for the latest task to be executed. With this solution I can simply add a server and it will automatically be configured. No changes to the deployment script, or trigger a push deploy is needed.

Note: I don’t have the goal to build a high scalable solution for hundreds of servers.

Here is an architecture overview:

The deployment controller doesn’t know anything about the deployment agents, only the agents know about the deployment controller. I use NuGet.Core to get packages from the Artifact Repository. I also use the Microsoft.Web.Xdt library for configuration files transformations. The deployment agent is a thin service, it’s the workflow activity the agent will execute that handle all the things that should happen within the agent.

Deployment Workflow

 

I decided to use Microsoft Workflow to specify the deployments steps that should be performed. I decided to use workflow because of its design editor and to easily visualize the deployment steps. I have created a Sequence, Parallel, InstallNuGetPackage and Agent activity. The reason I have created a custom Sequence and Parallel activity is to make it pass global variables to child activates. Variables that can be used when a configuration file is transformed, or by a PowerShell script. Variables in Workflow is scope based, I needed to make the variables global within the scope and to its children. I was kind of disappointed that Workflow didn’t pass parent variables to a child activity. The agent activity is the one that will serialize all of its activity and add it to the queue for an agent to capture. The InstallNuGetPackage is used to download a package from the Artifact Repository, and handle the execution of PowerShell scripts and configuration transformation. I will probably add more activity, such as Installing Service, creating MSMQ queues, creating web applications etc. Activity that will reduce the PowerShell script.

Here is a simple deployment workflow:

Note: The above workflow will only demonstrate that an agent activity can perform activity in Sequence and the above will also perform InstallNuGetPackages in parallel.

 

The goal with the workflow is to make it easier for other team members to setup deployment script by drag & drop of activities, and also have a visualized overview of the script.

If you would like to follow my progress or know when I post a new blog post, please follow me on twitter: @fredrikn

Read More

Can’t Reproduce?

Mark Shuttleworth, multimiljonären (mijardär?) bakom Canonical och Ubuntu har stängt* buggen som startade Ubuntu: When Canonical founder Mark Shuttleworth began shipping Ubuntu nine years ago, he created “Bug #1.” Its title was “Microsoft has a majority market share,” and Shuttleworth said “[t]his is a bug which Ubuntu and other projects are meant to fix. Idag […]

Read More

Microsoft Sommarkollo 2013

Magnus Härlin och Fredrik Normén från oss på Squeed kommer att prata på Microsoft Sommarkollo den 25/6 i Göteborg, “ASP.Net, det senaste, det heta och en titt in i framtiden”. Kommer att bli mycket spännande, vem vet kanske lite Angular.js också! Boka reda nu för att vara säker på att du får en plats: http://www.microsoftsommarkollo.se/index.php/2013/05/asp-net-det-senaste-det-heta-och-en-titt-in-i-framtiden/

Read More