How to allow for multiple types deployment?

Question

In my search for the meaning of life, I stumbled upon a blog post that mentioned that your deployment strategy is not your architecture, it is simply an implementation detail, and as such we need to design for allowing different deployment patterns, whether you want to deploy your system to 1 node or multi-node, or another type of structure.

Do the latest versions of Visual Studio provide some kind of flexibility (besides azure) to be able to deploy services in a variety of strategies?

For example, let's say I have a solution

    Acme Solution
    --Acme Startup Proj
    --Acme Service A.csproj
    --Acme Service B.csproj
    --Acme Service C.csproj

I want to be able to deploy this entire solution as 1 solution, or I would like to be able to deploy 3 separate binaries, one for each microservice.

AcmeServiceA.exe
AcmeServiceb.exe
AcmeServicec.exe

What does Visual Studio give you in terms of flexibility of deployment configuration?

Answer 1

Deployment techniques will vary with which technologies your app is built. For the sake of an example, I'm going to assume we're dealing with web services or sites.

You've specified two deployment scenarios: deploying a single project (e.g. microservice), and deploying all projects (full rollout). Let's start small...

Deploying an individual project

The main thing to plan for is that each deployable atom (this could be a project or a service + DB backend... something as small as you would prefer not to split it into smaller deployments).

For web projects (either it be Web API projects or other types), Visual Studio's built-in options can be generally summarized as: WebDeploy, Azure, and now with .NET Core, Docker images. I'm not going to go into the details of each, because those are separate questions. But I may refer to some details for you to research if they sound interesting (I'm more familiar conceptually with WebDeploy, so I'll refer to that a lot; but I'm not advocating for or against it).

If you were using WebDeploy for example, you could have each project produce a WebDeploy Package. (Again, look this up for more details on how to do it). This package can be crafted to contain a file payload (the site/service files) as well as a database payload, or other subatoms using the WebDeploy provider model. Visual Studio has pretty decent support for this scenario, and there is documentation on it.

Or you could generate a Docker image. From my understanding (and lack of experience with Docker as yet), if you wanted to deploy your web service and database, they ought to be in separate containers. You'll soon find yourself building these yourself outside of VS. That's not a bad thing, Docker sounds very flexible once you get the hang of it; but you are leaving the IDE for this.

Either way, now you can deploy the atomic package. This was the easy part.

Deploying the solution

So, you've got lots of these atomic deployment packages. How do you roll them all out?

Well, at this point VS doesn't provide a lot for you. And it's hard to justify what VS should do here. Almost every organization is going to come up with slightly different rules. Do you deploy from your CI? Do you create packages and deploy them to different environments in your release pipeline? Or do you do it in the cloud and hotswap environments (like Azure deployment slots)?

A VS native solution has to be either extremely configurable (and hence extremely complicated), or it will be too simple to fit most customers' needs. (As an aside, the initial support for WebDeploy back in VS2010 errored on the first of these. It was extremely configurable, and very difficult for customer or even the product team to wrap their heads around all of the possible scenarios. Source: I was the QA for that feature once upon a time.)

Really at this point you need to determine how and when you rollout your deployments. You need something to orchestrate each of these deployments.

VS generally orchestrates things with MSBuild. Again, I'm not advocating this as your orchestration platform (I actually dislike it for that... it's ok for your project configuration, but IMO not a good fit for task management), but if this is what you want to use, it can work. It's actually pretty simple if you're using it for to the Web Project scenario. You can build your solution and use the parameter /p:PublishOnBuild=true. If you are using WebDeploy to directly publish, you're done! If you're creating WebDeploy Packages, then you still need to push those, but at least you've created them all at once.

If you are using WebDeploy Packages, they will each generate a script to use for publishing. There are ways of passing in different WebDeploy parameters as well, so you can reuse the same package (build output) to publish to different environments. However, you'll have to write your own script to combine all of these into one megalithic deployment.

Ditto for Docker as well. You may get a set of images, but you still need something to orchestrate publishing all of them. Tools like Kubernetes can help you rollout, or in the event of issues, rollback.

There's also more generic orchestration platforms like Octopus Deploy.

How Unsatisfying!

Yeah, it kind of sucks that there isn't an out-of-the-box solution for large scale deployments. But if there was, it wouldn't work for 95% of teams. Most of what VS does provide is enough for an individual or very small development team to get their code to their servers. Any larger of a team, and you'll get better mileage out of building a system that is tailored for how your team operates. There are plenty of tools out there and none of them work perfectly in all cases. Find one that works for you, and you'll be fine. And in the end, it all comes down to pushing files and running scripts. If you don't like one system or tool, you can try another one.

Answer 2

If you are looking for an improved deployment experience in Visual Studio, check out Flexera's InstallShield Limited Edition in-box solution (ISLE, http://blogs.msdn.com/b/visualstudio/archive/2013/8/15/what-s-new-in-visual-studio-2013-and-installshield-limited-edition.aspx). ISLE is a great solution for those customers looking for added capabilities not found in Visual Studio Installer Projects, such as TFS and MSBuild integration, support for creating new web sites and ISO 19770-2 Tagging support, etc.

VS2015: https://marketplace.visualstudio.com/items?itemName=VisualStudioProductTeam.MicrosoftVisualStudio2015InstallerProjects

VS2017: https://marketplace.visualstudio.com/items?itemName=VisualStudioProductTeam.MicrosoftVisualStudio2017InstallerProjects

With the Setup and Deployment project templates you can choose to package all assemblies in the solution or each one individually as MicroService's using Setup, Web, CAB or Merge Module Projects:

Then choose which assemblies are included:

Answer 3

It really kind of depends on the exact use case how to achieve that requested some kind of flexibility and on your definition of an acceptable level of such a flexibility.

Taking your example with these three different executables as separate microservices (Service A, B, C) and as a complete service (Startup) in the context of Web.Api you could do the following:

1. Each project (Service A, B, C) can be designed as a separate OWIN self hosted executable (as outlined in Use OWIN to Self-Host ASP.NET Web API 2) and provide one or more endpoints to be exposed.

2. The main project (Startup) could also be an OWIN self host or a regular IIS Web.Api application that references the three projects (Service A, B, C) and load their respective endpoints in its own Startup routine (and optionally additional endpoints of iteself).

You can then use a separate configuration project in Visual Studio (or an external project in a completely different environment) and make use of deployment technologies like Puppet, Chef, or whatever to deploy according to your scenarios.

Your code would then be unaffected by the deployment you are actually wishing to perform and that respective configuration would be managed separately.

If this does not answer your question or if I have misunderstood your question, could you please clarify it and give more details?

Answer 4

When we talk about the meaning of life, here are the two cents about it by a deployment (install) specialist :-)- the answer is (seems :-) long, but it will contain specific information where to look for every point..

First of all, let's state that deployment is NOT a nobrainer, though many developers would like to see it like that (and as a deployment specialist, I observe quite often stakeholders in the software development process actually thinking like this- simple put, deployment is kinda forgotten until the day before shipment :-)

Compare it with coke for a bold and simple example. The "developers" produce the liquid, but it is quite easy to realize here, that the job isn't done yet. :-)

Visual Studio itself has not really support for deployment strategies. Based on several areas of deployment as mentioned in the following list there are of course a lot of technologys, some by Microsoft, helping with that. What I would do is building setup bundles for different customers or scenarios which install subsets of services like client/server scenarios or others (see no. 3. in the following list.)

---

Second, as you may have seen from other answers, deployment is not deployment. Partly, this depends if one sees deployment as just producing binary files by MSBuild or deploying to a test system or deploying to the customer, e.g. by updating the productive web site or producing DVDs or uploading executables to the update web site...

There are several different areas which sure have relations, but every numbered area is large and complicated enough to have own specialists for it:

1. Deployment seen as part of architecture has to deal with source and binary structures and entities, e.g. project and binary structure (how much .exe, .dll files, how are their dependencies, variation planning. => As you mentioned, you are here in the area of (Visual Studio, etc.) solutions, projects, as well namespaces, especially in the WCF area you have contracts, etc., you have (POC#) interfaces, etc. You have Nuget or other tools to resolve and manage dependencies. .NET has to offer the concept of the assembly to deal with this, the architecture, e.g. if to deploy interfaces and contracts in own assemblies, how to deal with client/server scenarios, how the assemblies depend on each other, is up to you and architecture..

   • Concerning services, there is an interesting subtask how to host services, They can be hosted on a web server, they can be selfhosted in an .exe, they can be hosted with IIS or OWIN, etc. Links for more information:

   • Selfhosting in WCF: https://msdn.microsoft.com/en-us/library/ee939340.aspx

   • Selfhosting in a Windows service, here with SignalR: https://code.msdn.microsoft.com/windowsapps/SignalR-self-hosted-in-6ff7e6c3

   • Hosting with OWIN: https://en.wikipedia.org/wiki/Open_Web_Interface_for_.NET

2. Deployment as part of a local Windows or other operating system integration: You have to think about, in which system directories you have to place certain files or data generally. You have to think about shared dlls, shared data, project data, temporary data, user specific data, registry, file system, Windows logo requirements, best practices, service configuration, etc.

3. Deployment as a process of creating setups, own installations, which, besides other things, accomplishes the needed actions mentioned in 2- with additional tasks like graphical installation front-end (setup GUI), license acknowledgement, what's new section, selection of optional components (just think of Visual Studio setup), uninstall/repair/modify possibilities, and so on.

4. Deployment as a devops process, e.g. part of continuous integration , continuous delivery and/or continuous deployment. Here are two main points: Technically, to have a defined process which is doing things mentioned in 2. and 3. (or alternatively web deploy steps) automatically as part of the build process ("post-build step").
   This can include creating setups or hierarchies of setups- or working without setups at all). The second is to enable testers, developers and managers (or even customers) to see at least every morning or even more often the already installed example of the last nightly or daily build, maybe with several deployment variants (client/server?, basic/prof?) or on different systems. You are here half in developer world, half in admin world. Here the main point is often not creating complicated setups like in 3. but primarily to define own "pack" and copy (and sign... etc.) processes, and to automate them as part of the development (and test and delivery) process. Puppet and Chef were already mentioned.

5. Deployment as web or cloud deployment (can also be the endpoint of a devops process)- others have said something about that, I will omit details here, but an important differentiation is, if you are talking about deployment to the customer or deployment to an intermediary test or staging system. Maybe one thing making this point worth to be mentioned additionally to devops, is that a deploy to online servers, server farms or a cloud has very own challenges.

6. Deployment seen primarily as an administrative process of distributing shippable, buyed and/or own programmed software to all the thousands of PCs in a company and it's daughter firms. there are of course special tools for this including update strategy, monitoring, license management and more. You are here in admin world, not in developer world anymore. Microservices will be a new and high challenge to admins which are mostly used to install and distribute "large" packages like MS Office or Oracle or whatsoever.

   • This topic is not so boring for developers as it seems. Primary because the two "worlds" of developers and admins are merging. And developers have to care about the customer view of "running the software in the real world". Devops is only the beginning. Everybody knows virtual machines, but now we have software defined networking, virtual apps, virtual server farms, the cloud, etc. You can define a deployment architecture by dependendies without any programming just by configuration. So deployment should be part of your application architecture, but mostly it isn't (enough). In fact until now the admin view is nearly nowhere integrated with the view of the software producers/developers. Concerning Microsoft, there is a lot of work done here by the Windows team, esp. in the server product line, and that was never really strategically coordinated with the developer team AFAIK (this is probably valid for EVERY software shop until now :-)

Currently, a lot of people publishing related to devops or the continuous buzzwords are not very experienced with setups. Building setups can be seen as a special technology among the other necessary steps.

Given that you are interested in knowing more about 3. (setups) : If you don't want only to copy executables, but to have the functionality of full setups, which do more work than just copy, part of setup strategy can be to have bundle setups (sometimes called suite setups or bootstrapper setups) with own selection features. They can call the underlying small setups e.g. for your microservices.

Visual Studio itself has not longer an own support for the more sophisticated setup types like MSI, and especially never had for grouping setups to bundles, what can be one possible solution of deploying a bunch (or variants of bunches) of services- VS has e.g some support for "ClickOnce" deployment, but this has been made more for database ("smart") clients than for services or even microservices. ClickOnce: https://msdn.microsoft.com/de-de/library/31kztyey.aspx

A replacement for the lack of "real" setup creation in Visual Studio can be the WiX toolset which is an Open Source project formed by Microsoft employees. Or InstallShield Express (which is a free, but a limited variant of the commercial ones). With both you can create full MSI setups which are maybe the most sophisticated setup type in the windows setup zoo.

a) Of course there are other setups types besides MSI (aka Windows Installer), they are from third party vendors which are more or less proprietary but more simple: , e.g. Nullsoft - NSIS and InnoSetup.
I will not give links for creating single MSI setups because they can be easily found with the given links of creating bundles of MSI setups in the next lines:

b) The tool for creating setups that select and install other (defined subsets of underlying) in the Wix "world" is called "Burn":

Creating bundles of setups with Burn: http://wixtoolset.org/documentation/manual/v3/bundle/

Special (paid) support for this you can get for example from the founder of WiX who created a company especially for this: https://www.firegiant.com/wix/tutorial/net-and-net/bootstrapping/ Rob Mensching, the founder, can be found here on SA as well answering dedicated questions.

c) InstallShield Suite setups: Another is the already mentioned tool InstallShield, but for this you will need their InstallShield Premium variant which costs bucks:
http://helpnet.installshield.com/installshield21helplib/helplibrary/SteCreatingSuites.htm

d) Setup-Factory : https://www.indigorose.com/setup-factory/

e) I am sure, many people would advise to take a look into Docker. Virtual applications are not only setups, but they isolate themselves in the "installed" state from other apps like a sandbox. See for example https://docs.docker.com/docker-for-windows/

f) The list would be not complete, if I would not mention APP-V as virtual application installation technology which shares some but not all features with docker. But these technologies are not really made for orchestrating multiple deliveries but to deliver just one app. And Microsoft has defined a new setup type called AppX. Especially you have to differ, if you want to create "legacy" (full) desktop applications for Windows where MSI setups are the known technology for or store apps which are the new type since Windows 8 (aka Universal Windows apps aka Windows Store apps aka modern apps aka Metro apps).

AppX: https://msdn.microsoft.com/en-us/library/windows/desktop/hh446767(v=vs.85).aspx AppX targets a more simple setup type than MSI.

Universal Windows apps (UWP): https://learn.microsoft.com/en-us/windows/uwp/get-started/whats-a-uwp

For anything more detailed we have to know more of your requirements.