Sitecore DevOp Series – Part 3 – Setup and Configure TDS

In previous blogs of the DevOp series, we have installed a local instance Sitecore and configured our VS for the sample Sitecore project. In this blog, we will add Team Development for Sitecore (TDS ) projects to our VS project. The blog series is aimed at newer audience and developers who are setting up CI for the first time.

Why do I need TDS ?

TDS is a great product for developing Sitecore projects. If you have not used it before, I would strongly recommend using it for your existing and new projects. The TDS seamlessly integrates with the Visual Studios and gives you a lot of options to manage Sitecore items. One of the main feature of TDS is that you can store Sitecore Items as serialized files on your local machine and push them to the source control repository. The development team can get the latest from source control, sync changes using TDS commands and update their local instance of Sitecore databases. This feature alone will save you and your team hundreds of hours development time.

The TDS can also automatically package up the data template changes for installations and deployments. It also gives you many more advance features for continuous integration which you can read on their official site.

Setup TDS Projects

Technically, you can have just one TDS project for your website and also configure it for continuous deployment. However, inspired from Pavel’s blog and personal experiences,  in this series we will be setting up the following 4 TDS projects

  1. MyProject.TDS.Core
  2. MyProject.TDS.Master
  3. MyProject.TDS.Master.Content
  4. MyProject.TDS.Master.System

TDS.Core project will be used to store any Sitecore’s Core database items like custom profile item or a new field type information.

TDS.Master project will be used to store Sitecore’s Master database items that are more specific to developers like templates, renderings, layouts, placeholder settings.

TDS.Master.Content project will be used to store Sitecore’s Master database items under the ‘Home’ node. If you have multiple sites, you can have multiple TDS.Master.Content projects, one for each site.

TDS.Master.System project will be used to store Sitecore’s Master database items under ‘System’ node. The ‘System’ node have items like ‘Workflow‘ or ‘Publishing Targets‘ or ‘Sitecore Powershell Scripts‘ that should be part of the source control, but we don’t necessarily need to deploy them every time.

All of the above TDS projects are optional and are meant for simplifying the deployment process as you will see in coming posts of the blog series.

Step 1 : Create TDS.Core project

Assuming that you have installed TDS for the correct version of your Visual Studios, right-click on the VS solution and click on ‘Add New Project’ and the select ‘TDS Project with Wizard’.

Give the project appropriate name like ‘MyProject.TDS.Core‘ and click on ‘Next’

my-project-tds-core1

On the next screen, set up the following parameters :

  • Sitecore Web Url
  • Sitecore Deploy Folder
  • Source Web Project
  • Sitecore Database
  • Check the install Sitecore connector

The example configuration is shown below:

 

my-project-tds-core2

Once you click Ok, a success message will pop-up and the new project should be added as below:

my-project-tds-core3

Step 2 : Create TDS.Master project

This step is similar to the previous one, except there is a small change when adding parameters on screen 2 of the wizard.

Right-click on the solution and click on ‘Add New Project’ and select ‘TDS Project with Wizard’. Give it the name as ‘MyProject.TDS.Master’ as shown below:

my-project-tds-master1

On the second screen, make sure to change the ‘Sitecore Database’ option to ‘master‘ and also un-tick the ‘Install Sitecore Connector’.

my-project-tds-master2

Click Ok and the project should be add to the VS solution:

my-project-tds-master3

Step 3 : Add TDS.Master.Content Project

Repeat the configurations for step 2, only changing name of the project

MyProject.TDS.Master.Content

my-project-tds-master-content1

Same parameters as that of step 2, un-tick the ‘Install Sitecore Connector’ check box:

my-project-tds-master-content2

New project should be added on clicking Ok

my-project-tds-master-content3

Step 4 : Create TDS.Master.System Project

Repeat the configurations for the step 2, only change the name of the project

MyProject.TDS.Master.System

my-project-tds-master-system1

Same parameters as that of step 2, un-tick the ‘Install Sitecore Connector’ check box:

my-project-tds-master-system2

New project should be added on clicking Ok.

my-project-tds-master-system3

Step 5 : Get Sitecore Items

So now we have our TDS projects installed and we would like to populate these propjects with Sitecore items. When you will right-click on the ‘MyProject.TDS.Master‘ project and try to click ‘Get Sitecore Item‘ command, it will be disabled as shown below:

my-project-tds-master-get-content1

This is intended behavior as during setup of the TDS.Master project we left the ‘Install Sitecore connector‘ option un-ticked. If you have left it ticked during the project wizard, it will create a different Access GUID for each project which will be incorrect for our 4 TDS project setup.

Copy Sitecore Access GUID

As we have only installed the connector for TDS.Core project, we need to get the ‘Access Guid’ and copy it across to the TDS.Master Project

For this, right-click on the TDS.Core project’s properties and navigate to the ‘Build’ tab. At the end of the panel, you will find ‘Sitecore Access GUID‘ property. Select and copy that value

my-project-tds-master-get-content2

Paste Sitecore Access GUID

Navigate to the build tab of ‘TDS.Master‘ project’s properties build tab and now tick the ‘Install Sitecore Connector‘ box, a random GUID will appear. Delete that GUID and paste the same GUID as for the TDS.Core project as shown below:

my-project-tds-master-get-content3

Verify connection for TDS.Master

Once you have copied over the same GUID, click on the ‘Test‘ button at the end of the panel to verify the connection details. They should all be green as shown below:

my-project-tds-master-get-content4

Get Sitecore Items

Once you have ran the test, give it like 60 seconds and try to ‘Get Sitecore Items’ again, now all options should be available and you should be be to get Sitecore items as shown below:

my-project-tds-master-get-content5

If you select to add the standard ‘Sample Item’ template and all its children by right-click options, it should appear as below:

my-project-tds-master-get-content6

Repeat the Copy/Paste GUID process for TDS.Master.Content and TDS.Master.System

my-project-tds-master-get-content7

Once you have repeated the process for all TDS.Master.* projects and populated them with Sitecore items, your VS solution should look like this :

my-project-tds-master-get-content9

That’s it for now, in the next post we will configure Sitecore Glass Mapper, TDS with T4 templates and auto-code generation for the Sitecore data templates.

Stay tuned.

Thanks

 

Related Blogs

  1. Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?
  2. Part 2 – Setup and Configure Visual Studio Sitecore Project
  3. Part 3 – Setup and Configure TDS
  4. Part 4 – Setup Sitecore Glass
  5. Part 5 – Setup Source Control (Git)
  6. Part 6 – Setup QA Server, DB server and CI server
  7. Part 7 – Setup Continuous Integration using Team City
  8. Part 8 – Setup Slack Notifications with TeamCity and Bitbucket

 

Sitecore DevOp Series – Part 2 – Setup and Configure Visual Studio Sitecore Project

This is the second part of the Sitecore DevOp Series. In this blog post, we will setup a Visual Studio project for a brand new Sitecore, create a local publish profile and add XML config transforms. The blog series is aimed at newer audience and developers who are setting up CI for the first time.

Step 1 : Create Visual Studios Project

At the time of writing this blog post, I have used VS 2015. Open Visual Studio, navigate to File > New > Project and within templates select Templates > Visual C# > Web > ASP.NET Web Application project. Make sure the .NET Framework is set to 4.5.2 or above. Optionally you can un-tick the ‘Application Insights’ box if you are not using it:

my-project-vs-452

The VS project physical location should be setup somewhere other than the installed instance of Sitecore. For the purpose of the blog, it will be at C:\projects\MyProject where as the installed Sitecore instance will be at C:\inetpub\wwwroot\myproject\website. If you are not sure which version of .NET to use with your Sitecore instance, please check out the compatibility matrix.

Step 2 : VS Project Type

Select ‘Empty’ template project but tick the ‘MVC’ box under the core references. If you are not using Azure, un-tick host in the cloud option as shown below:

my-project-vs2

Step 3 : Exclude and Delete Files

Once the project is created by the wizard, it will also installed some standard configuration files which we will overwrite in the next step. For now, please exclude the following files from the project and also delete their physical version:

  1. Global.asax
  2. Global.asax.cs
  3. Web.Config
  4. Web.Config.Debug
  5. Web.Config.Release

my-project-vs3

Step 4 : Copy and Include Files

Navigate to the installed directory that was created automatically by the Sitecore Installer. In this case it will be C:\inetpub\wwwroot\MyProject\Website and copy the following files into current project:

  1. Default.aspx
  2. Default.css
  3. Default.js
  4. Global.asax
  5. Web.config
  6. Webedit.css

Include these files via the VS project ‘Include File’ options

Step 5 : Add Sitecore References

Navigate to the bin folder of the installed directory that was created automatically by the Sitecore Installer. In this case it will be C:\inetpub\wwwroot\MyProject\Website\bin and copy the following files in a folder called ‘references’ (or lib or another folder of your choice)

  1. Sitecore.Kernel
  2. Sitecore.Client
  3. Sitecore.Logging
  4. Sitecore.Mvc
  5. Sitecore.NVelocity
  6. Sitecore.Updated
  7. Sitecore.Zip
  8. Sitecore.Analytics

my-project-vs4

Reference these files via the VS project’s  ‘Add Reference’ option

Recently, Sitecore has also released NuGet feed for distribution purposes, you may use that as well as per their documentation.

Step 6 : VS Custom Projects

Sitecore has recently released and recommended Helix Framework for overall design of Sitecore projects. It is strongly recommended for any future projects to be based around these design principles. However, for simplicity purposes and keeping in mind the target audience, we can just add the following class library projects:

  1. MyProject.Domain (It can contain auto-generated data model for the templates and other models)
  2. MyProject.Sc (It can contain Sitecore specific customization code like workflow, pipelines etc)

my-project-vs5

Once both projects are added, the overall VS project should look like below:

my-project-vs6

Step 7 : Setup Local publish profile

The next step will be to setup a local publish profile for the project, as the installed directory location was different then the VS project location.

Create a new custom publish profile as ‘local’ from the ribbon option

my-project-vs7

 

Select ‘File System‘ for the publish method drop-down and select the target location of the installed directory as shown below:

my-project-vs8

Select ‘Debug‘ option from the configurations drop-down so you can debug the code locally after deployments. This should only be chosen for local deployments, for QA and other environments, ‘Release‘ should be selected.

my-project-vs9

Preview all your settings and hit publish, upon successful publish you should see the following message:

my-project-vs10

Step 8 : XML config transforms

We need to create environment specific config files which we can then transform to correct config settings during the automated deployment.

For this purpose, install VS SlowCheetah extensions and then you right-click to add any new configuration files. For example, after I right-click on the Sitecore.config I see the following options:

my-project-vs11

Once you click ‘Add Transform’ new config file (Sitecore.local.config) will be added in the project. This is where environment specific settings can be updated and upon publish they will be transformed. This alone will save you a lot of time of managing different config settings for different environments.

 

my-project-vs12

A sample XML transform file for replacing the dataFolder location could be:

<?xml version="1.0" encoding="utf-8" ?>
   <sitecore xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
       <sc.variable name="dataFolder" value="C:\Inetpub\wwwroot\myproject\Data\" xdt:Transform="Replace" xdt:Locator="Match(name)"/>
   </sitecore>

Although, you don’t need to specify this variable for local deployment, but you will when it comes to QA, UAT and production deployments. You may want to repeat the same process for connectionstrings.config file.

More information about XML transformation can be found on MSDN website

Once the initial setup project is done, we are ready to add configure TDS projects which will be discussed in the next post.

Stay tuned.

Thanks

Related Blogs

  1. Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?
  2. Part 2 – Setup and Configure Visual Studio Sitecore Project
  3. Part 3 – Setup and Configure TDS
  4. Part 4 – Setup Sitecore Glass
  5. Part 5 – Setup Source Control (Git)
  6. Part 6 – Setup QA Server, DB server and CI server
  7. Part 7 – Setup Continuous Integration using Team City
  8. Part 8 – Setup Slack Notifications with TeamCity and Bitbucket

Sitecore DevOp Series – Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?

Over past few years with Sitecore, I have worked with various technical teams to deliver Sitecore powered websites. Sometimes we all are located in the same office, but most of the times, we are located in different offices and in different countries.

Part of my job role is to make sure that all of the development work is ready for testing (QA) at agreed time intervals (end of development sprint).

Quality Assurance (QA) is an important part of delivering any project and when a QA team member discovers a defect in the system, my usual conversation with the QA team member goes something like this:

QA : “Naveed, this feature is not working on the QA site?

Me: “Hey John (the fellow developer who did the original code), can you check why is it not working on the QA website?

John: “I am not sure, all my code is checked-in and it works on my machine as expected!

If you have worked on a Sitecore project, you must have had a similar conversation. The reason we have this conversation is because the deployment was done “manually”, which is an unreliable and error-prone process. A file or a package can very easily be missed during the deployment or a wrong version of a file can be deployed.

Moreover, the “manual deployment” process is very time consuming and it will increase the development cost. A typical Sitecore project may have numerous environments like QA, UAT and Production and each environment may have multiple CM and CD servers. As each server may be configured differently and may have different set of config files or config settings, the “manual” deployment process becomes even more complex. So you can see the “manual deployment” process, which started with one QA server, can quickly become very time consuming (read costly).

In an ideal world, what we would like for a Sitecore powered project, is that if someone has written a piece of code and/or updated Sitecore items, it should work on their local machine, it should work on the QA server (or any other non-production servers) and it should work on the production servers. The effort required should be minimal and nothing should be missed during deployment process. And for audit purposes every deployment should be documented.

Automated Deployments (Continuous Integration)

The best possible way to achieve this to make sure that the deployment process is fully automated from start to finish. All pieces of code and related Sitecore items should be packaged, deployed and published with “one single click” of a button.

To achieve our goal of ‘one-click-deployment’, there are tools and techniques available today (well actually since few years) which we can configure for continuous integrations and automated deployment. This will solve our 2 main issues:

  • Our deployments will be reliable and consistent.
  • We will be saving time and money.

If you are still doing “manual deployments” within your organization, it is time to re-think and re-evaluate this process.

The purpose of the blog series is to target newer audience/developers/tech teams, who have never done CI before and take them through a ‘hello world’ type of continuous integration process (automated builds).

Server Architecture

For the automated deployments to work, the code udpate workflow should be as following:

1- Developers should work locally and check-in their code in the source control (use of feature branches, fix branches and pull-requests is highly recommended)

2-Continuous Integration server  builds and deploys the code to the QA server

The diagram below shows a typical server architecture for the local and QA environment:

my-project-qa-setup

 

The important thing to notice here is that Sitecore items (templates, layouts, renderings, sub layouts) can be sourced control using tools such as TDS or Unicorn tools. I will be referring to TDS within the blog series.

As a minimum, automated deployment process for the above mentioned QA environment will require three servers :

1 – QA Server – Sitecore CM/CD combined on same instance

2- DB Server – Sitecore SQL server databases and MongoDB

3 – CI Server  – TeamCity server for continuous build and deployment

In theory, you can use only server, but it will have performance issues for the QA website.

Tools Stack

The tools stack that I will use to demonstrate the process in the series is as below :

  1. Sitecore CMS 8.2
  2. Visual Studios 2015
  3. SQL Express 2014
  4. Hedgehog Team Development (TDS) (licenses required)
  5. Sitecore Glass
  6. SlowCheetah
  7. Git
  8. TeamCity
  9. Chocolatey
  10. Sitecore Ship

There are other tools that can be used for the dev ops, for example, Jenkins can be used in place for TeamCity. Octopus Deploy can be used solely for deployment purposes, leaving CI server to do one job, that is to build the latest code. The decisions to choose the correct stack tools should be dependent upon your business requirements.

For simplicity purposes and for the newer audience, I will be using TeamCity for both build and deployment processes. Once you get confident with simpler process, you can enhance it/supplement it with different tools as you see fit for your project.

Assumptions for the blog series

I will assume that you starting a new project and have a blank machine/servers with only Windows on it or it is your first day at your new job. You can skip any of the below steps if you feel necessary:

  1. Download and install latest version of Visual Studios on your local machine or VM. At the time of writing this post, I have used VS 2015.
  2. Download and install latest version of SQL Express on you local machine or VM. At the time of writing this post, I have used SQL Express 2014
  3. Download and install TDS for VS2015 (requires commercial license, you can use 30 days evaluation for free)

I will also assume that you have RDP and FTP access to the QA, DB and CI servers as described in the architecture diagram to setup continuous integration.

As mentioned before, the blog series is intended for the  ‘new audience’, developers or teams who have never done CI integrations. The series will be pretty detailed and a single blog post will not do any justice, therefore it is divided into smaller logical parts of the setup process all of which will be posted in coming weeks.

The next part of the series are:

Sitecore DevOp Series – Part 2 – Setup and Configure Visual Studio Project

Sitecore DevOp Series – Part 3 – Setup and Configure TDS

Sitecore DevOp Series – Part 4 – Setup Sitecore Glass

Sitecore DevOp Series – Part 5 – Setup Source Control (Git)

Sitecore DevOp Series – Part 6 – Setup QA Server, DB server and CI server

Sitecore DevOp Series – Part 7 – Setup Continuous Integration using Team City

Sitecore DevOp Series – Part 8 – Setup Slack Notifications with TeamCity and Bitbucket

 

Local Sitecore Installation

Assuming it is a new project, the first part will be to install a blank Sitecore on your local machine. There are various options for installing Sitecore on your local machine

  1. You can install using the official .exe installer
  2. You can install using the official .zip file and attaching databases
  3. You can install using SIM module.
  4. You can install using Sitecore Rocks VS extension.

Every developer will have their preferred way for installing a brand new Sitecore website, I mostly prefer official .exe installer and will be referring to this in future posts as well.

Please refer to the official documents for installing the basic Sitecore website and do not forget to do the post-deployment steps.

 

my-project-local

 

For the purpose of the blog series, the project will be called ‘myproject‘. Once you have installed Sitecore locally and you can navigate the link http://myproject.local on your machine you are ready for the next step. By default, the installation location will be C:\inetpub\wwwroot\myproject\

In the next part of the series we will setup a VS project locally, configure a publish profile and add XML config transforms.

Stay tuned.

Thanks.

 Related Blogs

  1. Part 1 – Continuous Integration – Why your Sitecore project deployments must be automated ?
  2. Part 2 – Setup and Configure Visual Studio Sitecore Project
  3. Part 3 – Setup and Configure TDS
  4. Part 4 – Setup Sitecore Glass
  5. Part 5 – Setup Source Control (Git)
  6. Part 6 – Setup QA Server, DB server and CI server
  7. Part 7 – Setup Continuous Integration using Team City
  8. Part 8 – Setup Slack Notifications with TeamCity and Bitbucket