In this article we are going to implement a step by step approach for Continuous development and Integration model with a distributed version control system, artifactory, package manger and a package repository for third party libraries. We will also explore the real use of a private Artifactory in the model. In the first part, we will implement the model without a private Artifactory and then in the second part of the article we will optimize the model with the use of a private Artifactory.
Tools used
Following tools are used for the implementation
| Development IDE | Visual Studio 2015 |
| Repository | Git |
| Version Control | Git-TFS |
| Package Manager | Nuget |
| Public Package repository | Nuget Library |
| Artifactory | JFrog |
| Build tool | TFS |
| Deployment tool |
Steps for Continuous development and Integration
- Set up Git repository.
- Connect to the repository from Git client.
- Setup Artifactory.
- Identify the eligible packages for the private Artifactory.
- Create and publish packages into the Artifactory.
- Organize your application to use all binaries and publicly available third parties repository through the private artifactory in time of building the project.
- Build source code.
- Deploy artifacts.
Implementation
Create a Team project and setup Git repository
Create a team project on Visual Studio Team Services/TFS interface. Sign up and create your team project. Select Git from the version control options:
Project will be created successfully.
Navigate to the project. You will be presented with the home page for the project. Navigate to CODE page for the project:
Once you create a new project as with the option Git as the version control a new Git repository will be created by default. The CODE page shows information for the newly created empty Git repository. The page displays the Clone URL. This URL will be used further for cloning the created repository from any Git client.
You can also create new repositories from repository dropdown.
Clone the repository from Git Client
We have created our Git repository. Now it’s time to add source into the repository. To add code, you'll first need to clone your repo to your local machine using Visual studio Git plugin. Once you have a local clone, you can start adding code to your repo.
The remote repository can be cloned in the following way:
Add source code to the repository
Create a new project in Visual Studio from the team explorer. If you want to add an existing project, open the project from Team Explorer.
Create a new console application.
A console application will be created. Notice that your new solution is showing up on the Home page under Solutions.
In Team Explorer, navigate to the Changes page.
The new application appears under the Included Changes section. Enter a commit message and click the Commit and Push button to commit the changes to your local repo then push your changes to Visual Studio Team Services.
Your code will be successfully pushed into local repo and TFS.
Note:
On the Changes page, you will be prompted if you haven't configured your username or email address. If you haven't previously used Git on this computer, you may have to configure your username and email address.You can also access the Git Settings page from the Settings page in Team Explorer, under the Git section, Global Settings.
View the added code from Git remote repository
View the changes pushed to the server by clicking on CODE menu in the TFS interface.
Integration with public package library (Nuget)
Enterprise application contains lots of publicly available libraries (e.g. Log4Net, jQuery etc.). In the past, we used to check-in those libraries into version control system along with our source code. With the new CI and CD model we are not going to check-in those libraries into source control. Rather we will include few configuration files in our application and instruct our project to get required binaries from a public package library such as Nuget. In the time of building the code in a build server (e.g. TFS), the build process will use the configurations files to download the libraries with appropriate version from a package library location.
In order to implement the approach, let’s include a third party library in our sample application.
Install Nuget package for third parties from VS2015 IDE
Let’s add the third party library Elmah in our application. Following are the steps to add and install the Nuget package:
Go to Tools – Option. Then Nuget Package Manager – Package sources or Click on the "Manage Nuget Packages" from the Project Add Reference context menu. By default, the NuGet Package Manager window will refer to the all package source located at "Nuget.Org" as shown in image below.
Go to Tools – Option. Then Nuget Package Manager – Nuget Package Manager for Solution. Search for the third party library Elmah.
Select the library for your project and install.
The third party library Elmah will be successfully installed and automatically referred to the project. A file named packages.config will be created automatically. The file has the package id and version number for the third parties or binaries.
="1.0"="utf-8"
<packages>
<package id="elmah" version="1.2.2" targetFramework="net452" />
<package id="elmah.corelibrary" version="1.2.2" targetFramework="net452" />
</packages>
We will check-in this file into the source control instead of the binaries (e.g. ELmah). In time of build, the build process use this file to get package id and version number for required binaries for the project.
Create a file named Nuget.config just below the solution and add the following lines.
="1.0"="utf-8"
<configuration>
<solution>
<add key="disableSourceControlIntegration" value="true" />
</solution>
<config>
<add key="repositoryPath" value=".\packages" />
</config>
<packageSources>
<add key="nuget.org" value="https://www.nuget.org/api/v2/" />
</packageSources>
</configuration>
Nuget.config only has effect when integrating with TFS. The DisableSourceControlIntegration value restricts check-in of the binaries into the source control. PackageSources defines the source URL for Nuget package sources. Currently, we are accessing binaries from the Nuget public package library (https://www.nuget.org/api/v2/).
Note:
For building the code successfully in TFS, it is very important to create the Nuget.config at solution file (.sln) level. Later, in the Build solution step, the TFS build process looks for a folder named "packages" in the project root according to <HintPath>. Non-existence of the folder under root results in build failure.
Add few lines of code for Elmah implementation.
class Program
{
static void Main(string[] args)
{
Elmah.MemoryErrorLog log = new Elmah.MemoryErrorLog();
log.Log(new Elmah.Error(new Exception("Hello ELMAH!")));
List<Elmah.ErrorLogEntry> list = new List<Elmah.ErrorLogEntry>();
log.GetErrors(0, 100, list);
foreach (Elmah.ErrorLogEntry err in list)
{
Console.WriteLine(err.Error.Exception.Message);
}
Console.ReadLine();
}
}
Build the code in VS2015.
Commit and push the modified solution into Git repository.
Build the project in TFS
Now it’s time to build our project in TFS. We will pull our source code from Git repository and binaries from Nuget public package library. Let’s crate a build definition accordingly.
Build Definition
Access the TFS interface from VSTS and navigate to the created project. Select Build menu.
Click on Create New Build definition button to create a new build.
Select Visual Studio as the build template.
The repository and the master branch will be selected by default. Verify the same and proceed.
Create the build.
Now we are required to add build steps. Our sample application will get the Elmah Nuget package for public library and then build the source code. So, we will define just two build steps for the time being:
- Nuget Restore.
- Build solution.
Add the build step for Nuget Restore.
- Provide the path of the solution or the Package.config file.
- Provide the path to the Nuget.config file.
Select the Build Solution step. Provide the path of the solution file (.sln).
Save the build.
Select Queue New Build and queue the build for execution.
Project will be built successfully.
Observe the log for Nuget Restore build step. The Nuget package Elmah is pulled from the Nuget public package library (https://www.nuget.org/api/v2/).
So, we have implemented continuous integration and development approach to some extent with Nuget public library. We have isolated our source code from binaries and pulled the binaries from public package library in time of build. Still, the approach leave us with few open questions:
- The packages resides in the public library. Lots of users are downloading the packages from the public package repository. A typical project may need 5-10 packages from Nuget repository. How I can reduce network traffic and optimize my build?
- What to do if the public package library repository goes down or if there is an issue with the network?
- How to control what people in the organization download from public package libraries instead of using an in-house custom library?
- How to control which public libraries should be searched for packages at first?
- How to use an in-house custom package without exposing it to public package library?
All these open questions lead us to the direction to use an intermediary private repository between developers and public package libraries (Nuget). We can use Artifactory to act as an intermediary and solve the above mentioned issues.
In the next part, we will optimize our continuous integration and development model with the use of JFrog Artifactory.
