Sitecore DevOp Series – Part 6 – Setup QA Server, DB server and CI server

Happy New Year 2017!!!

This is part 6 of the Sitecore DevOp series. Previously, we have installed a local instance of the Sitecore site, configured VS project, configured TDS, configured Sitecore Glass and added our project to source control. In this post we will install blank Sitecore site on QA server and setup QA website’s Web Deploy publish profile with our VS project. The blog series is aimed at newer audience and developers who are setting up CI for the first time.

Assumptions

I am assuming that you have RDP access to at least 3 Windows based server within your local network/organization. From the diagram below, the QA server will host CM and CD part of the website (qa.myproject.local). The DB server will host SQL server and MongoDB databases. The CI server will host the TeamCity server (teamcity.myproject.local). The blog series is based upon Sitecore version 8.2

my-project-qa-setup

This is only a QA setup diagram, for staging and production environments, it is strongly recommended to add separate CD and CM servers.

Step 1 : Configure DB Server

The first step will be to configure the DB Server.

  1. Download and install SQL express
  2. Download and install MongoDB

For compatibility purposes of SQL and MongoDB versions with your version of Sitecore, please check the KB page https://kb.sitecore.net/articles/087164

Setup MongDB to run as Window Service and open the relevant ports from the windows firewall utility on the DB server for incoming and outgoing connections.

Alternatively, if you are new to MongoDB and not sure how to configure and install MongoDB, you can also use mlab free Sandbox environment which gives 500 MB of free storage. You can setup your MongoDB servers following this post and just update the connection strings for the QA server config file.

Step 2 : Install Sitecore Databases

RDP to the DB server and install Sitecore using the official installer. During the wizard steps, select the ‘New Instance‘ option and then select ‘Database Only‘ option:

db-1

Follow the installation wizard as per the installation guide and complete it.At the end, you should have the following 5 SQL databases:

Core, Master, Web, Analytics and Session

db-2

Step 3 : Configure QA Server

RDP to the QA server and using the official installer, select ‘New Instance’ and ‘Client Only’ option:

qa-client-1

Follow the installation wizard and complete all steps. Update your local DNS server or your own hostfile so that http://qa.myproject.local point towards the QA server. Login via the CMS interface using default admin credentials and complete the post-deploy steps from the installation guide.

Step 4: Configure Web Deploy on QA Server

Web Deploy is an IIS extension/utility that provides ability to deploy new updates on the website using IIS only. If your server does not have this utility for IIS, install it via standard installer.

Once installed complete the following steps:

  • Open IIS, navigate to the website which corresponds to http://qa.myproject.local
  • Right-Click on the website and generate WebDeploy Publish Profile

 

qa-client-2

 

  • Select a local computer user like (machinename\WDeployAdmin) and give this user admin rights by making it of the local admin groups.
  • Save the publish profile as a file on the desktop and copy it to your local machine.

As a reference, follow the installation guide.

Publish Profile for QA server

Open the VS project and import the profile for the QA site. You can open the publish profile using notepad and re-name the profile name. Make sure you can validate the connection for the publish profile. If you are getting errors, please resolve them using this post before proceeding further:

qa-client-3

XML transform files for QA profile

Add XML transform files for the new ‘qa‘ publish profile. As a minimum, add transforms for ConnectionStrings.config and Sitecore.config. You will be required to update values for connection strings and dataFolder:

qa-client-6

Try to publish to the QA server via VS project, if everything has been done correctly, you should see the successful message. Navigate to http://qa.myproject.local and ensure everything is correct. If there are errors, resolve them before proceeding further.

qa-client-7

This ensures that Web Deploy is configured correctly and there are no permissions or network issues. This is only the initial part, what we really want to build and deploy from our CI server and not from a local machine of a developer.

Step 5: Setup CI server (TeamCity and Plug-ins)

Assuming it is a brand new Windows server with nothing installed, you need to download and install the following the TeamCity server

  1. TeamCity latest stable version
  2. Visual Studio Community Edition
  3. Team Development Server for Sitecore (TDS)
  4. Chocolaty
  5. Git

All of the above tools have a standard Windows base installer and they follow ‘Click Next’ type wizard to complete. Once all these utilities are installed, we will begin setting up the CI server in next blog post.

Stay tuned.

Thanks

Related Blogs

  1. Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?
  2. Part 2 – Setup and Configure Visual Studio Sitecore Project
  3. Part 3 – Setup and Configure TDS
  4. Part 4 – Setup Sitecore Glass
  5. Part 5 – Setup Source Control (Git)
  6. Part 6 – Setup QA Server, DB server and CI server
  7. Part 7 – Setup Continuous Integration using Team City
  8. Part 8 – Setup Slack Notifications with TeamCity and Bitbucket
Advertisements

Sitecore DevOp Series – Part 5 – Setup Source Control (Git)

This is part 5 of Sitecore DevOp series. Previously, we have installed a local instance of the Sitecore site, configured VS project, configured TDS, configured Sitecore Glass for auto-code generation.

In this blog, we are going to add our VS project to a source control. As a developer or as an organisation you have many options available to select your source control like SVN, TFS, or Git . The choice of source control depends upon your business requirements or your client’s requirements.

For the purpose of the blog, I will be using bitbucket.org with repository type as Git. The blog series is aimed at newer audience and developers who are setting up CI for the first time.

Assumptions

I am assuming that the person reading this post has never used Git or Bitbucket before, so I will be doing a step-by-step description.  If you have not used Git before please spend 1 hour reading about it’s basic operations and install a Git client like Tortoise Git or Source Tree. I will be using Tortoise Git for the blog post. If you are a git pro, you can ignore this blog post as all we are doing in the post is setting up our project with BitBucket.

Step 1 : Create a new repository

If you don’t have an account with bitbucket.org, create a free account. Then from Repositories top-menu create a new repository. For the blog series, I created ‘MyProject‘ repository as shown below:

bit-bucket-1

once you have created the repository, you will get an admin interface that will look like this

bit-bucket-4

 

Step 2 : Create repository locally

Once you have created repository on bitbucket.org, you want push your changes to the server. Before you can do that, you need to right-click the ‘MyProject‘ folder and select ‘Git create repository here‘ as show below: (you will get these options if you have installed Tortoise Git)

bit-bucket-2

Step 3: Commit changes locally

The next step is to commit your changes locally. Few things to remember

  • Only commit what you think is required to be source controlled
  • Do not commit run time changes like /bin folder or /debug folder
  • Do not commit cache items as they will be local

For example, using tortoise git and right-clicking on the ‘MyProject‘ folder, if I do not want to add MyProject.TDS.Core/bin folder and all the files in that folder, I will just add that folder path to .gitignore file as shown below:

bit-bucket-6

As a minimum, I will also add a .gitattributes file and add the following settings as a minimum:

*.cs text=auto diff=csharp 
*.html text=auto
*.htm text=auto
*.css text=auto
*.scss text=auto
*.sass text=auto
*.less text=auto
*.js text=auto
*.sql text=auto

*.csproj text=auto merge=union 
*.sln text=auto eol=crlf merge=union

*.item -text

Step 4: Push changes to the Server

Once you have committed locally, you then need push changes to the server. But before you can push changes, you need to set the remote project URL from your client’s settings

For example, if this the project URL:

https://bitbucket.org/myusername/myproject.git

Then From right-click on ‘MyProject’ folder > Tortoise Git > Settings > Git > Remote add URL as shown below:

bit-bucket-7

Once you have set up the URL, and you try to push changes, the server will ask for credentials. Upon successful authentication your project files will be pushed to the server

bit-bucket-3

Once you have pushed your changes if you will navigate back to the admin interface of your project, you should see new folders and files created as shown below:

bit-bucket-5

Our project is now ready for continuous integration. In the next blog we will set up the servers and configure them for CI.

Stay tuned.

Thanks

Related Blogs

  1. Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?
  2. Part 2 – Setup and Configure Visual Studio Sitecore Project
  3. Part 3 – Setup and Configure TDS
  4. Part 4 – Setup Sitecore Glass
  5. Part 5 – Setup Source Control (Git)
  6. Part 6 – Setup QA Server, DB server and CI server
  7. Part 7 – Setup Continuous Integration using Team City
  8. Part 8 – Setup Slack Notifications with TeamCity and Bitbucket

Sitecore DevOp Series – Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?

Over past few years with Sitecore, I have worked with various technical teams to deliver Sitecore powered websites. Sometimes we all are located in the same office, but most of the times, we are located in different offices and in different countries.

Part of my job role is to make sure that all of the development work is ready for testing (QA) at agreed time intervals (end of development sprint).

Quality Assurance (QA) is an important part of delivering any project and when a QA team member discovers a defect in the system, my usual conversation with the QA team member goes something like this:

QA : “Naveed, this feature is not working on the QA site?

Me: “Hey John (the fellow developer who did the original code), can you check why is it not working on the QA website?

John: “I am not sure, all my code is checked-in and it works on my machine as expected!

If you have worked on a Sitecore project, you must have had a similar conversation. The reason we have this conversation is because the deployment was done “manually”, which is an unreliable and error-prone process. A file or a package can very easily be missed during the deployment or a wrong version of a file can be deployed.

Moreover, the “manual deployment” process is very time consuming and it will increase the development cost. A typical Sitecore project may have numerous environments like QA, UAT and Production and each environment may have multiple CM and CD servers. As each server may be configured differently and may have different set of config files or config settings, the “manual” deployment process becomes even more complex. So you can see the “manual deployment” process, which started with one QA server, can quickly become very time consuming (read costly).

In an ideal world, what we would like for a Sitecore powered project, is that if someone has written a piece of code and/or updated Sitecore items, it should work on their local machine, it should work on the QA server (or any other non-production servers) and it should work on the production servers. The effort required should be minimal and nothing should be missed during deployment process. And for audit purposes every deployment should be documented.

Automated Deployments (Continuous Integration)

The best possible way to achieve this to make sure that the deployment process is fully automated from start to finish. All pieces of code and related Sitecore items should be packaged, deployed and published with “one single click” of a button.

To achieve our goal of ‘one-click-deployment’, there are tools and techniques available today (well actually since few years) which we can configure for continuous integrations and automated deployment. This will solve our 2 main issues:

  • Our deployments will be reliable and consistent.
  • We will be saving time and money.

If you are still doing “manual deployments” within your organization, it is time to re-think and re-evaluate this process.

The purpose of the blog series is to target newer audience/developers/tech teams, who have never done CI before and take them through a ‘hello world’ type of continuous integration process (automated builds).

Server Architecture

For the automated deployments to work, the code udpate workflow should be as following:

1- Developers should work locally and check-in their code in the source control (use of feature branches, fix branches and pull-requests is highly recommended)

2-Continuous Integration server  builds and deploys the code to the QA server

The diagram below shows a typical server architecture for the local and QA environment:

my-project-qa-setup

 

The important thing to notice here is that Sitecore items (templates, layouts, renderings, sub layouts) can be sourced control using tools such as TDS or Unicorn tools. I will be referring to TDS within the blog series.

As a minimum, automated deployment process for the above mentioned QA environment will require three servers :

1 – QA Server – Sitecore CM/CD combined on same instance

2- DB Server – Sitecore SQL server databases and MongoDB

3 – CI Server  – TeamCity server for continuous build and deployment

In theory, you can use only server, but it will have performance issues for the QA website.

Tools Stack

The tools stack that I will use to demonstrate the process in the series is as below :

  1. Sitecore CMS 8.2
  2. Visual Studios 2015
  3. SQL Express 2014
  4. Hedgehog Team Development (TDS) (licenses required)
  5. Sitecore Glass
  6. SlowCheetah
  7. Git
  8. TeamCity
  9. Chocolatey
  10. Sitecore Ship

There are other tools that can be used for the dev ops, for example, Jenkins can be used in place for TeamCity. Octopus Deploy can be used solely for deployment purposes, leaving CI server to do one job, that is to build the latest code. The decisions to choose the correct stack tools should be dependent upon your business requirements.

For simplicity purposes and for the newer audience, I will be using TeamCity for both build and deployment processes. Once you get confident with simpler process, you can enhance it/supplement it with different tools as you see fit for your project.

Assumptions for the blog series

I will assume that you starting a new project and have a blank machine/servers with only Windows on it or it is your first day at your new job. You can skip any of the below steps if you feel necessary:

  1. Download and install latest version of Visual Studios on your local machine or VM. At the time of writing this post, I have used VS 2015.
  2. Download and install latest version of SQL Express on you local machine or VM. At the time of writing this post, I have used SQL Express 2014
  3. Download and install TDS for VS2015 (requires commercial license, you can use 30 days evaluation for free)

I will also assume that you have RDP and FTP access to the QA, DB and CI servers as described in the architecture diagram to setup continuous integration.

As mentioned before, the blog series is intended for the  ‘new audience’, developers or teams who have never done CI integrations. The series will be pretty detailed and a single blog post will not do any justice, therefore it is divided into smaller logical parts of the setup process all of which will be posted in coming weeks.

The next part of the series are:

Sitecore DevOp Series – Part 2 – Setup and Configure Visual Studio Project

Sitecore DevOp Series – Part 3 – Setup and Configure TDS

Sitecore DevOp Series – Part 4 – Setup Sitecore Glass

Sitecore DevOp Series – Part 5 – Setup Source Control (Git)

Sitecore DevOp Series – Part 6 – Setup QA Server, DB server and CI server

Sitecore DevOp Series – Part 7 – Setup Continuous Integration using Team City

Sitecore DevOp Series – Part 8 – Setup Slack Notifications with TeamCity and Bitbucket

 

Local Sitecore Installation

Assuming it is a new project, the first part will be to install a blank Sitecore on your local machine. There are various options for installing Sitecore on your local machine

  1. You can install using the official .exe installer
  2. You can install using the official .zip file and attaching databases
  3. You can install using SIM module.
  4. You can install using Sitecore Rocks VS extension.

Every developer will have their preferred way for installing a brand new Sitecore website, I mostly prefer official .exe installer and will be referring to this in future posts as well.

Please refer to the official documents for installing the basic Sitecore website and do not forget to do the post-deployment steps.

 

my-project-local

 

For the purpose of the blog series, the project will be called ‘myproject‘. Once you have installed Sitecore locally and you can navigate the link http://myproject.local on your machine you are ready for the next step. By default, the installation location will be C:\inetpub\wwwroot\myproject\

In the next part of the series we will setup a VS project locally, configure a publish profile and add XML config transforms.

Stay tuned.

Thanks.

 Related Blogs

  1. Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?
  2. Part 2 – Setup and Configure Visual Studio Sitecore Project
  3. Part 3 – Setup and Configure TDS
  4. Part 4 – Setup Sitecore Glass
  5. Part 5 – Setup Source Control (Git)
  6. Part 6 – Setup QA Server, DB server and CI server
  7. Part 7 – Setup Continuous Integration using Team City
  8. Part 8 – Setup Slack Notifications with TeamCity and Bitbucket