Bringing Agile into SharePoint - ALM

SharePoint development platform, provides the capabilities to develop, deploy, and update customizations and custom functionalities for the SharePoint sites. The activities that take advantage of these capabilities all fall under the category of Application Lifecycle Management (ALM).

“ALM is a continuous process of managing the life of an application through governance, development and maintenance.”

With the options like BCS, Client OM, Sandbox solutions, WCF based Service Applications and Claims based authentication SharePoint has also significantly evolved as application development platform which opens up a number of extensibility options for the developers. With the power and flexibility to use these tools and customizations the question arises is how to effectively and efficiently manage the activities taking into account the lifecycle of application starting from requirement gathering to release management with consistent quality.

The entire process involves setting up SharePoint with visual studio, TFS, Build automation, Deployment and automated testing.

Setting up DEV environment

To take the full advantage of development capabilities, SharePoint is installed on all development machines. We use scripted installation of SharePoint (SPAutoInsaller) which helps to maintain a similar environment for all development machines and helps the team to configure SharePoint easily.

TFS as a source control

With Visual studio as a development tool, the integration to source control is much easier as the IDE is easily integrated to the TFS. The team creates and works on team projects, checks in the code directly from Visual Studio to TFS.

TFS for build automation

CI builds are configured to ensure that every code that is checked in passes the build with the configured build standards. On every code check-in the build server to pull down the latest source code, compile the code, run any static analysis rules, executes all unit tests and integration tests on the latest code and report any errors along the way. This is a great way of ensuring that even if the developers locally don’t run the static analysis rules or unit tests they will be run here and also ensures that all source code is checked in and is working as expected.

 

 

TFS for Code analysis

Code analysis rules help teams to ensure that coding is done according to the standards defined and are followed without fail. The default set of rules are built into Visual Studio which can also be tweaked to meet the specific needs of the teams. Apart from the built in rules which are common to all projects, additional rules like SPDisposeCheck that analyzes the code to see whether all SPWeb and SPSite objects are disposed correctly can be added to the rules. These rules are run on both Visual Studio and TFS to ensure continuous quality.

Automated deployment

Automated deployment is triggered as part of the build process, or as an unattended service to deploy solution packages. The deployment process is a combination of batch files, PowerShell commands and xml configuration files. It is a great way to update the test machines with latest version of the software every time a new version is checked in.

Automated unit, smoke and regression testing

Testing is also configured as part of the CI build process or a nightly build process depending on the teams requirements. Most of the unit tests are executed on every check-in build and regression tests are executed as part of the nightly builds. Team focuses on creating and maintaining a clean set of test cases that help the team get an early feedback on the quality of the product after every release to the test environment.

 

Bringing Agile into SharePoint - CI and DTAP

Creating and maintaining a DTAP (Development-Test-Acceptance-Production) environment helps the project teams and IT department ensure faster and quality delivery at a lower cost. It also reduces the risk of breaking other applications when side by side deployment of components is facilitated in higher environments. The teams get early feedback on the compatibility and performance of the software in Production by simulating the deployment on a production like environment (A). All stakeholders (including the customer) can be involved in the acceptance of the feature before the feature is deployed in Production by following the DTAP street.

The code checked in by the team is rebuild on a daily basis with unit test and deployed daily to a test environment (CI box). All changes to CI and subsequent environments are deployed using wsp's and PowerShell scripts. A fully automated CI process is implemented by a combination of XML configuration files and PowerShell scripts. Having a process like this will also ensure that installation on all environments including development have a similar process.

Environments and purpose

Development (D)

Standalone VM's with self-contained SQL server, SharePoint, Visual Studio and related tools for developers to create and debug solutions.  Developers work on code checked into TFS and are responsible for maintaining a passing build on every check-in. These servers are also used as quick demo environments to end users and building proof of concepts.

The development machines are configured by using PowerShell scripts which makes it easy to configure SharePoint on machines by team members who have minimal knowledge of SharePoint.

Test (T)

This environment is mainly used by IT department. Teams make use of this environment to fine tune installations before moving up to the next farm in the street. This environment has a stable release so testers can perform better integration testing against a specific release build. Project teams use this environment for demos to end users and have installations multiple times before going to A.

Acceptance (A)

This a production like server where the IT department performs final validation of the software before going to production. This environment is not used for final testing, but to use as a stage to get final approval from the PO before going to production. 

Production (P)

This is the final phase of the development process. The installations are all fully automated and have the content to serve the end users!!!

Bringing Agile into SharePoint - Quality check

One of the major problems with SharePoint development is that the framework is not designed with testability in mind. It is not easy to replace the SharePoint object model using Mocks/ Stubs which makes practices like unit testing more difficult in SharePoint development.

The main focus will be to use a combination of best architectural practices and isolation frameworks to achieve best results to do Agile in SharePoint. For e.g. isolating SharePoint calls using a Repository pattern makes it easy to mock a repository object in the above layers to isolate SharePoint object model. Building SharePoint solutions using XP techniques helps the teams in faster development, predictable deliveries, better quality, less stress and happy stakeholders

Repositories

Repositories act as mediators, or interface layers, between different parts of an application. Making use of the repository pattern the SharePoint object model can be exposed using a and repository which will be responsible for using the SharePoint API for querying and updating SharePoint list items.

By using a combination of Specification and data mapper pattern with Repositories will allow the team to centralize the SharePoint list access logic and conversion of types. This provides a simplified access mechanism of list data and provide a layer of abstraction that makes unit testing with emulators/mocks much easier

 

MVP

Separating the logic for the UI from the code that handles user events, fetching data from repositories by using an implementation of MVP allows developers to replace the actual view with a mock object for unit tests.

SharePoint Emulator

With the availability of SharePoint Emulators the isolation of the SharePoint API is much easier compared to the old way of isolating using the Moles or Fakes framework.

In order to incorporate emulators into existing tests you should wrap the relevant code in a SharePointEmulationScope, which accepts EmulationMode enum as a parameter. It is enabled by default which in turn performs run time interception of all Microsoft.SharePoint.dll calls.

[TestMethod]

[TestCategory("Data access layer tests")]

[Description("Test to verify the data mapper functionality")]

public void BuildFromSPListItem_should_create_a_new_object_with_populated_values()

{

    using (new SharePointEmulationScope(EmulationMode.Enabled))

    {

        var dataMapper = new MyObjectDataMapper();

        const string customDescription = "Custom Description";

        using (var emulatedSite = new SPSite("http://localhost:8081/MySite"))

        {

            var listId = emulatedSite.RootWeb.Lists.Add("MyList", "Custom list", SPListTemplateType.GenericList);

            var list = emulatedSite.RootWeb.Lists[listId];

            var listitem = list.Items.Add();

            listitem["Title"] = "mytitle";

            listitem["CustomDesc"] = "Custom Description";

            var myObject = dataMapper.FromSPListItem(listitem);

            Assert.IsTrue(myObject.CustomDesc == customDescription);

        }

    }

}

Smoke tests

Smoke tests are quick, lightweight test or set of automated tests, that confirm that the basic functionality of the sites are working as expected. We use the smoke tests to verify whether the basic standards of internet sites are adapted and are working as expected in all the environments. The smoke tests are executed from a set of configuration files that specify the details of the sites and environments to run against.

By having proper smoke tests, the team can ensure that the site works with the basic functionalities after every upgrade or patches are done on the farm.

Automation tests

The team also creates a set of UI driven automated tests that cover the functional testing of the UI controls. This helps us verify that the whole application including the user interface is functioning correctly. Teams create Coded UI tests that help remove the need for manual regression testing of a functionality. The Coded UI tests simulates the user experience of clicking through the UI controls to verify things are working correctly.

Brining Agile into SharePoint - Benefits

Learn early

Doing sprints help the team identify issues early at the project life cycle. This helps the team find things that are good or bad for the team or process early in the development stage rather than waiting till the end of the project to get feedback. This way the teams Learn early from the mistakes and can better themselves in the upcoming sprints.

Potentially shippable software at the end of every sprint.

The intention of every sprint is a potentially shippable software that is of immediate use to the user. By delivering a product that can be used and tested will also help the team receive early feedback on the work done. The team can then make changes to the product based on these feedback which will eventually result in a more value delivered to the end users.

This will also help the team perceive shippable obstacles and issues early in the development state compared to the waterfall model where issues related to release of software is often identified at the end.

Kaizen mindset

The scrum artifacts like daily standups, sprint reviews, retrospections help the team to focus on continuous improvement through inspect and adapt. The team manges to improve by identifying and managing the impediments during the process.

Better visibility and improved productivity

The team focuses on keeping the progress of the project visible to the entire world. Everything the team is busy with is displayed on the tailboard. The task board shows the progress of the stories or tasks, spikes, bugs and the status of each item.

Burn down charts facilitate as symptom checkers for the team and help them to identify the smells during the progress of the sprint. For e.g a flat burndown shows that either the team is working on a lot of stories at time or nothing is getting done completely in the sprint.

The velocity charts helps business envision and evaluate the productivity of the team after every sprint.

Bringing Agile into SharePoint - Combination

SharePoint offers several flexible approaches that enable rapid development and deployment of business processes through its out-of-the-box features which are also easily customizable. A SharePoint developer can leverage a lot of this out of box functionality and combine it with .NET skills to create highly functional business solutions quickly, easily and at a fraction of the cost of traditional solutions.

A combination of SharePoint and Agile can be used to deliver good solution to the customers with improved speed up overall delivery time with better focus on developer productivity.

So what are the benefits that SharePoint lends to Agile?

A lot of out of the box functionality:
Most of the client requirements map directly onto features provided from a default provision of SharePoint. With a wide choice of from web templates to workflows most of the features SharePoint has supports the vision for the customers’ requirements.

Rapid prototyping:
SharePoint comes with an abundance of features and tools that help developers quickly build business solutions even if they don’t have a strong background in web development. You can easily build, customize and deploy solutions using SharePoint. It’s very simple to create a rough working model of what the customer is looking for that helps the team to ensure that they can deliver what the customer really wants using SharePoint.

Easy customization:
With SharePoint, you have an option of OOTB features with which the development team has the ability to build custom applications and components with ease. Along with the help of proper tooling the administrators can customize the entire SharePoint experience with your his/ her organizations branding.

Easy deployment and administration:
Administrators can easily access and manage features, system settings, monitor farms, perform backup and restore content, security through the Central Administration Console all in a single location.

I'll write more on the combination of Agile and SharePoint in the upcoming posts and we'll also see how to address some of the challenges of Agile development practices in SharePoint like unit testing, TDD etc.