Note: This article is part of series of articles showcasing different approaches & variations to adjust cache size of the SharePoint 2013 Distributed Cache service using PowerShell commands – Set-AFCacheHostConfiguration and Update-SPDistributedCacheSize.

This is 2nd article in this series and provides step by step guide for SharePoint Admins to configure Distributed Cache size using Set-AFCacheHostConfiguration command and high level guidance on why and when you should use this command.

• Previous Article in this series – Adjust Cache Size of the SharePoint 2013 Distributed Cache Service Part I – Overview
• Next Article in this series – Adjust Cache Size of the SharePoint 2013 Distributed Cache Service Part III – Using Set-Update-SPDistributedCacheSize Command

Note: This article is part of series of articles showcasing different approaches & variations to adjust cache size of the SharePoint 2013 Distributed Cache service using PowerShell commands – Set-AFCacheHostConfiguration and Update-SPDistributedCacheSize.

This is 3nd and last article in this series and provides step by step guide for SharePoint Admins to configure Distributed Cache size using Set-Update-SPDistributedCacheSize command and high level guidance on why and when you should use this command.

Previous Article in this series -

• Adjust Cache Size of the SharePoint 2013 Distributed Cache Service Part I – Overview
• Adjust Cache Size of the SharePoint 2013 Distributed Cache Service Part II – Using Set-AFCacheHostConfiguration Command

Thanks for everyone who has made to my session at the SharePoint Fest Chicago 2013. I was really surprised with turnout and full jam packed session. I knew it would interest most of attendees as building News application or Knowledge center in corporate intranets are one of the most common use cases for SharePoint.

I have built three different kind of Corporate News Application in three different versions of SharePoint in last few years – SharePoint 2010, SharePoint Online 2010, and SharePoint 2013 and all of them built with different flavors and requirements. One common need for all environment were decoupled authoring and publishing features.

With SharePoint 2013, new publishing method called “Cross-Site Publishing” has made building Corporate News application easier than ever with decoupled architecture for both authors and end users. As promised, here is my session deck available through Slide Share. Feel free to download and reach out to me if you have any questions.

It was such a great time spent with the SharePoint Community and Slalom team!!!

Recently I came across interesting error “/_vti_bin/TaxonomyInternalService.json/ValidateTerms 500 (Internal Server Error)” while referencing managed metadata taxonomy field in the publishing page layouts. I have couple of managed metadata fields referencing managed metadata term store. While updating published pages, I was getting popup box with error message stating “There was an error processing the request”.

As you may have noticed, I have previously blogged an article – SharePoint March 2013 Public Update, Invalid FormDigest, Client.SVC ProcessQuery 403 Forbidden and it was dejavu for me. I remembered I had came across similar client side web service call issue on the March 2013 Update while editing published page articles. Based on previous experience, my first instinct was to look at the FormDigest and sure enough, that was the culprit.

Resolution => Upgrade to August 2013 CU => As I have done with the “Client.SVC ProcessQuery 403 Forbidden” issue, this issue seems to have been introduced into the product in the March 2013 CU for both SharePoint 2010 and SharePoint 2013. Upon upgrading my current environment to August 2013 CU, I was able to fix this issue.

Additional Resource - http://sharepoint-community.net/profiles/blogs/sharepoint-2010-publishing-page-error-there-was-an-error

As most of us in SharePoint industry aware, SharePoint March 2013 Update is the new baseline update for SharePoint 2013 farms and all the servers must be upgraded to March 2013 Update before patching to later Service Packs and Cumulative Updates.

Unfortunately, if you are using SharePoint Publishing Features to host articles and web content management features, you will encounter multiple issues and your authors won’t able to update articles in several scenarios. I have came across several issues – e.g. Calling CSOM APIs or referencing Managed Metadata Fields from Page Layouts.

Please review these two articles, understand the logic behind, and plan to patch your SharePoint farm to latest service pack or CU. At the time of writing this article – I was able to fix these issues by upgrading to the August 2013 CU.

• SharePoint March 2013 Public Update, Invalid FormDigest, Client.SVC ProcessQuery 403 Forbidden
• SharePoint 2013 March Update, Invalid FormDigest, Taxonomy Field in Publishing Page Layout, ValidateTerms 500 Error, and “There was an error processing the request” Error

Hope this will avoid several hours of headache for some of you out there.

Someone told me at SPC12 that Distributed Cache in SharePoint 2013 is User Profile Sync Service in SharePoint 2010. If you are seasoned SharePoint professional, you know that doesn’t bring smile on your face.

After reading Microsoft’s TechNet articles, one would assume that statement made at SPC12 was 100% on money. Reading TechNet articles on Distributed Cache, one should feel like they are walking on terrain of landmines and one wrong step could bring down the SharePoint farm and un-stabilize the SharePoint 2013 environment. If you still haven’t got the point, read this – Microsoft has raised first ever warning in SharePoint Products history stating “The Distributed Cache service can end up in a non-functioning or unrecoverable state if you do not follow the procedures listed in TechNet article. In extreme scenarios, you might have to rebuild the server farm“.

I don’t recall Microsoft ever issued warnings in this matter in previous four releases of the product which could bring down the farm, not even for User Profile Service in SharePoint 2010. Having worked on complex SharePoint 2013 farm over the last year, I have noticed that these warnings will surface it’s ugly head one time or another. Unfortunately many of core SharePoint 2013 features are dependent on Distributed Cache and disabling this service is not an option at this moment. If you ever notice that users can’t post anything in their my site news feed or people can’t login properly, either your Distributed Cache is lacking memory cache or Distributed Cache is dead and you may force to make necessary memory allocation changes for Distributed cache or revive dead Distributed Cache service.

Here are the major No-nos, Gotchas, or Warnings SharePoint admins should keep in mind regarding Distributed Cache service. Please note that most of these information directly derived from TechNet, Steve Peschka, and Josh Gavant‘s articles.

By default, 10% of Total RAM and 5% of Total RAM allocated to Cache Size to Distributed Cache Service

When SharePoint Server 2013 is installed, it assigns the Distributed Cache service 10 percent of the total physical memory on the server. The Distributed Cache service uses half of that memory allocation for data storage (also known as cache size, 5% cache size), and the other half of that memory allocation is used for memory management overhead. When the cached data grows, the Distributed Cache service uses the entire 10 percent of the allocated memory. http://technet.microsoft.com/en-us/library/jj219613.aspx

Never Administrator the Distributed Cache Service from the Services Snap-in in Administrative Tools

The Distributed Cache depends on Windows Server AppFabric as a prerequisite. Do not administer the AppFabric Caching Service from the Services window in Administrative Tools in Control Panel. Do not use the applications in the folder named AppFabric for Windows Server on the Start menu. Always administrator Distributed Cache using PowerShell or using Central Administration Services on Server Page. http://technet.microsoft.com/en-us/library/jj219613.aspx

Total memory allocation for Distributed Cache host shouldn’t be more than 16 GB

On a server that has more than 16 GB of total physical memory, allocate a maximum of 16 GB of memory to the Distributed Cache service. If you allocate more than 16 GB of memory to the Distributed Cache service, the server might unexpectedly stop responding for more than 10 seconds. If the cache host’s size is larger than 16GB/8GB, garbage collection could take long enough to cause a noticeable interruption for clients. If you require more memory, you can configure the Distributed Cache service to run on several application servers or dedicated Distributed Cache servers. As far as server specs, for dedicated Distributed Cache servers, your Virtual or Physical Server RAM shouldn’t be more than 24 GB (16 GB allocated to Distributed Cache and other for OS). – http://technet.microsoft.com/en-us/library/jj219572.aspx and http://blogs.msdn.com/b/besidethepoint/archive/2013/03/27/appfabric-caching-and-sharepoint-2.aspx

Dynamic memory is not supported in SharePoint 2013 environments nor App fabric servers.

The Distributed Cache service can run on either a physical or virtual server. When using virtualization, do not use Dynamic Memory to manage shared memory resources among other virtual machines and the Distributed Cache servers. The memory allocation for virtualized Distributed Cache servers must be fixed. http://technet.microsoft.com/en-us/library/jj219572.aspx

Plan to run Distributed Cache in Dedicated Mode

The Distributed Cache service can run in dedicated or collocated mode. When running in dedicated mode, the Distributed Cache service is started and all other services are stopped on the server. In collocated mode, the Distributed Cache service is running along with other services on the server. Dedicated mode is the recommended mode in which to deploy the Distributed Cache service. Review Steve Peschka’s article – http://blogs.technet.com/b/speschka/archive/2013/08/29/dedicating-servers-to-distributed-cache-in-sharepoint-2013.aspx

Developing code against the Distributed Cache of SharePoint 2013 is NOT supported

If you are using custom applications in SharePoint Server 2013 which use the AppFabric client APIs, or are creating custom caches, you should create a separate AppFabric cache cluster to support your custom applications. Do not use the AppFabric cache cluster supporting your SharePoint Server 2013 farm. Run your separate AppFabric cache cluster for your custom applications on separate servers from the servers dedicated to your SharePoint Server 2013 farm.  Microsoft has stated that additional named caches should not be deployed to the SharePoint AppFabric cluster (i.e. by using the New-AFCache cmdlet). If you need a cache for a custom solution, you’ll need to deploy a separate AppFabric cluster (or server) and create the cache there. Then point your solution at the external AppFabric cluster. There’s also no supported way to add your own cached items to SharePoint’s named caches. http://technet.microsoft.com/en-us/library/jj219572.aspx & http://blogs.msdn.com/b/besidethepoint/archive/2013/03/27/appfabric-caching-and-sharepoint-2.aspx

Install Windows AppFabric using SharePoint Server 2013 prerequisite installer

It is important to note that SharePoint 2013 RTM uses Windows AppFabric 1.1 for Distributed Cache Service. When the SharePoint Server 2013 prerequisite installer runs, it installs Windows Server AppFabric 1.1. This is the recommended approach for installing Windows Server AppFabric on a server that is running SharePoint Server 2013. If you already have Windows Server AppFabric 1.0 installed on the server (SharePoint 2013 Preview was based on Windows Server AppFabric 1.0) before you run the prerequisite installer, you must uninstall Windows Server AppFabric before you run the prerequisite installer. If an administrator decides to install Windows Server AppFabric manually, the administrator must install the CacheAdmin, CachingService, and CacheClient features, and use the /gac switch. For more information see Automated Installation (AppFabric 1.1 Caching) in the MSDN Library. http://technet.microsoft.com/en-us/library/jj219572.aspx

Plan to have Memory Allocation to all Distributed Cache Host are Same

You must ensure that the memory allocation assigned to the Distributed Cache service is the same on all servers that are running the Distributed Cache service. An administrator can change the memory allocation for the Distributed Cache service by using the Update-SPDistributedCacheSize or Set-AFCacheHostConfiguration cmdlet. http://technet.microsoft.com/en-us/library/jj219572.aspx and http://nikpatel.net/2013/09/24/adjust-cache-size-of-the-sharepoint-2013-distributed-cache-service-part-i-overview

To have Distributed Cache working on more than one servers, the first server with Distributed Cache needs to have its firewall set to allow for Inbound ICMP (ICMPv4)

If you are using more than one cache host in your server farm and they are not in same subnet, you must configure the first cache host running the Distributed Cache service to allow Inbound ICMP (ICMPv4) traffic through the firewall. If an administrator removes the first cache host from the cluster which was configured to allow Inbound ICMP (ICMPv4) traffic through the firewall, you must configure the first server of the new cluster to allow Inbound ICMP (ICMPv4) traffic through the firewall. http://technet.microsoft.com/en-us/library/jj219572.aspx  and http://blogs.technet.com/b/speschka/archive/2013/01/11/configuring-multiple-distributed-cache-servers-in-sharepoint-2013.aspx and http://blogs.technet.com/b/uktechnet/archive/2013/05/07/guest-post-distributed-cache-service-in-sharepoint-2013.aspx

Plan to update Distributed Cache memory allocation while updating physical or virtual RAM on Cache host

When you add physical memory to the server. The Distributed Cache service does not automatically recalculate the 10% memory allocation, so when you increase the total physical memory on the server, you have to manually increase the Distributed Cache service’s memory allocation. http://technet.microsoft.com/en-us/library/jj219613.aspx

Use skipRegisterAsDistributedCachehost parameter to configure Dedicated Distributed Cache Host in SharePoint Farm

When you run PSCONFIG or SharePoint Products Configuration Wizard to configure SharePoint on server in the farm, it will start Distributed Cache service by default and server will be added as a Cache Host. As an alternative to the default configuration, an administrator can install SharePoint Server 2013 without registering the Distributed Cache service on servers not intended to be part of the cache cluster. This can be achieved by using the skipRegisterAsDistributedCachehost parameter with the New-SPConfigurationDatabase or the Connect-SPConfigurationDatabaseWindows PowerShell cmdlets, or when running psconfig.exe at the command line. This parameter is optional. – http://technet.microsoft.com/en-us/library/jj219572.aspx

Use Add-SPDistributedCacheServiceInstance to add Cache Host in the Distributed Cache Cluster

If a Distributed Cache Service is missing for a particular server in Central Administration, you need to run “Add-SPDistributedCacheServiceInstance” from a SharePoint PowerShell window on the server it is missing on. It is important to note that adding server instance using Add-SPDistributedCacheServiceInstance resets cache size to 5%. Whether you use AppFabric or SharePoint cmdlets to modify cache host size, note that if you uninstall and reinstall the Distributed Cache Service Instance on a server (i.e. by running Remove-SPDistributedCacheServiceInstance and then Add-SPDistributedCacheServiceInstance) the cache host size will be reset to the default (5% of physical memory at time of installation). If removing and adding the cache service instance is part of your maintenance cycles, make sure to also modify the cache size afterwards if needed. – http://technet.microsoft.com/en-us/library/jj219613.aspx and http://blogs.msdn.com/b/besidethepoint/archive/2013/03/27/appfabric-caching-and-sharepoint-2.aspx

Plan to avoid Farm Account to run Distributed Cache Service

When AppFabric is installed as part of the SharePoint pre-requisites, the server farm account (e.g. Niks\sp_farm) is set as the service account of the AppFabric Caching service. The Distributed Cache service depends on the AppFabric Caching service. This will eventually trigger a violation of a Health Analyzer Rule. You must plan to change the service account of the AppFabric Caching service to a managed account. Primarily it can be same service account (e.g. domain user account – Niks\sp_svcapp) used to run all other Service Applications in the farm. On the side note, if you use AutoSPInstaller, this step is automatically configured by the AutoSPInstaller and Distributed Cache will run under domain service account other than Farm Account. http://technet.microsoft.com/en-us/library/jj219613.aspx – and http://blogs.technet.com/b/uktechnet/archive/2013/05/07/guest-post-distributed-cache-service-in-sharepoint-2013.aspx

Plan to stop gracefully and remove server from Distributed Cache cluster during patching and maintenance of the SharePoint server in the farmRun Stop-SPDistributedCacheServiceInstance –Graceful and Remove-SPDistributedCacheServiceInstance before shutting down the server for patching. Run Add-SPDistributedCacheServiceInstance after patching is done. It is also important to remember that adding server instance using Add-SPDistributedCacheServiceInstance resets cache size to 5% and you must resize the cache size using Update-SPDistributedCacheSize or Set-AFCacheHostConfiguration. Do not use this approach to stop the server if you need to update cache memory size. Instead manually start/stop Distributed Cache service from Service on Server page or start/stop service instance using PowerShell command. To adjust the memory cache for host, plan to use approached previously mentioned in my articles – http://nikpatel.net/2013/09/24/adjust-cache-size-of-the-sharepoint-2013-distributed-cache-service-part-i-overview/

SharePoint doesn’t provide high availability to Distributed Cache component of the farm

SharePoint (as of August 2013 CU) does not provide any high availability for its caches. It is important to note that Cached data stored in one server may not available on another Distributed Cache server in farm. If the server where that item has been stored in memory is lost or shut down ungracefully, that cached item will be lost. This is generally not a problem for cached items because they are authoritatively stored in the database except items in feed cache – Tags and document follow activities.

The Feed Cache depends on the Distributed Cache service. Tags and document activities are saved only to the Feed Cache. Tags and document activities are not persisted to content databases. When the Distributed Cache service is stopped, tags and document activities are lost. If these cached items are lost they won’t be able to be regenerated and will no longer appear in users’ feeds. To avoid losing items from the cache and/or having to retrieve them again, you can use the Stop-SPDistributedCacheServiceInstance cmdlet with the -Graceful switch. When the graceful shutdown of the Distributed Cache service method is used, all cache data is moved from one server to another server before the Distributed Cache service is stopped. For this to be effective, there must be space on the other servers to accommodate these items. Also note that if shutting down the entire cluster, such as to change the cache host size, there’s no way to avoid losing all of the feed caches items.

When the Distributed Cache service is started, repopulation occurs when the feed cache repopulation timer job runs. It is important to note that retrieving cached items all over again involves a performance hit. There could be interruptions and delays while the caches are being refilled. http://technet.microsoft.com/en-us/library/jj219613.aspx & http://blogs.msdn.com/b/besidethepoint/archive/2013/03/27/appfabric-caching-and-sharepoint-2.aspx

Additional References

• Overview of micro-blog features, feeds, and the Distributed Cache service in SharePoint Server 2013 – http://technet.microsoft.com/en-us/library/jj219700.aspx
• Plan for feeds and the Distributed Cache service in SharePoint Server 2013 – http://technet.microsoft.com/en-us/library/jj219572.aspx
• Manage the Distributed Cache service in SharePoint Server 2013 – http://technet.microsoft.com/en-us/library/jj219613.aspx
• Josh Gavant’s App Fabric Caching and SharePoint Part I and II – http://blogs.technet.com/b/mspfe/archive/2013/04/02/appfabric-caching-and-sharepoint.aspx
• Distributed Cache Fun in SharePoint 2013 – http://blogs.technet.com/b/sharepoint_republic/archive/2013/04/11/distributed-cache-fun-in-sharepoint-2013.aspx
• Sahil Malik’s Distributed Cache in SharePoint 2013 – http://www.code-magazine.com/Article.aspx?quickid=1309021
• Distributed Cache Service in SharePoint 2013 -http://blogs.technet.com/b/uktechnet/archive/2013/05/07/guest-post-distributed-cache-service-in-sharepoint-2013.aspx
• What every SharePoint admin should know about troubleshooting and managing the newsfeed & distributed cache -http://blogs.technet.com/b/sharepoint_ramblings/archive/2013/05/11/what-every-sharepoint-admin-should-know-about-troubleshooting-and-managing-the-newsfeed-amp-distributed-cache.aspx
• Points to consider with Distributed Cache on SharePoint 2013 – http://blogs.msdn.com/b/calvarro/archive/2013/08/29/points-to-consider-with-distributed-cache-on-sharepoint-2013.aspx
• Supportability of custom developments in SPS 2013 and APPFabric Cache – http://blogs.msdn.com/b/calvarro/archive/2013/06/05/supportability-of-custom-developments-in-sps-2013-and-appfabric-cache.aspx
• How to make SharePoint 2013 Distributed Cache service Semi High Available – http://mahmoudhamed.wordpress.com/2013/09/12/how-to-make-sharepoint-2013-distributed-cache-service-semi-high-available/
• Understanding SharePoint 2013 Distributed Cache Service, Microblogs and feeds – http://dayansameera.wordpress.com/2013/08/30/understanding-sharepoint-2013-distributed-cache-service-microblogs-and-feeds/
• AppFabric Distributed Caching and SharePoint 2013  – http://mysharedpoints.blogspot.com/2012/08/appfabric-distributed-caching-and.html
• SharePoint 2013: Distributed Cache Services – http://kerls.blogspot.com/2013/03/sharepoint-2013-distributed-cache.html
• Distributed Cache Service in SharePoint 2013 – http://www.c-sharpcorner.com/UploadFile/d2ee01/distributed-cache-service-in-sharepoint-2013/
• SharePoint distributed cache issues – http://www.andrevanhaaften.com/?p=94
• How to check memory allocation for distributed cache on SharePoint 2013?  – http://sp-vinod.blogspot.com/2013/03/how-to-check-memory-allocation-for.html

Good luck managing this beast!!!!!!

Here is the quick tip on how to turn on & off ULS Logs for Troubleshooting. This is mostly self-reminder note for me.

• Download ULSLogViewer from MSDN, this is one of my all time favorite tool which doesn’t install anything on the SharePoint servers  – http://archive.msdn.microsoft.com/ULSViewer/Release/ProjectReleases.aspx?ReleaseId=3308
• To ensure you can start tracing errors for troubleshooting, You must enable logging. By default you may have been logging Errors but plan to configure Verbose logging to ensure you have all the detailed information your would need for troubleshooting. Please note that verbose would fill up the disk space. Plan to disable as quickly as possible after troubleshooting is completed. To enable logging, Go to Central Admin – Monitoring -> Diagnostic Logging -> Select All categories or whatever you are interested -> Select “Verbose” in both least critical event to report event log & trace log. Although event log is not required, I prefer to log into event logs as well to ensure I have more than one tool for troubleshooting.
• As soon as you would turn on Verbose logging, it should start logging C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS (if that’s your default directory, plan to offload logs on the production environment).  You should see logs are filling up fast as soon as you turn on the Verbose logging.
• To view the errors, Open the logs from File-> Open From -> ULS for real time feed => C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\
  • You can find error & correlationid => Edit -> Find
  • You can filer log with correlationid => Edit -> Modify filter
• Plan to disable verbose logging or reset to Error logging once troubleshooting is done. To disable logging, Go to Central Admin – Monitoring -> Diagnostic Logging -> Select All categories or whatever you are interested -> Select “None” in both least critical event to report event log & trace log. It should stop filling up logs in the LOGs folder

Hope this helps both newbies and veterans out there. Enjoy!!!!!

I wanted to list out all the top SharePoint 2013 and Office 365 free video training resources available as of today. Microsoft has done outstanding job with SharePoint 2013 release and it’s unfortunate not many people know about all those resources released over the last year.

Hopefully this will help someone to have 1-stop place to get trained with latest and greatest in SharePoint field. I will try my best to keep the links up to date if it ever changes.

#1 => TechNet – SharePoint 2013 training for IT pros

http://technet.microsoft.com/en-us/sharepoint/fp123606.aspx

#2 => MSDN – Developer training | apps for Office and SharePoint

http://msdn.microsoft.com/en-US/office/apps/fp123626

#3 => Microsoft Virtual Academy – Tuning SQL Server 2012 for SharePoint 2013 Jump Start

http://www.microsoftvirtualacademy.com/training-courses/tuning-sql-server-2012-for-sharepoint-2013-jump-start#?fbid=2gpZeTlCxl7

#4 => Channel 9 MSDN – SharePoint Conference 2012 Session Videos

http://channel9.msdn.com/Events/SharePoint-Conference/2012

#5 => Office 365 Ignite Training

http://ignite.office.com/guides

#6 => Office 365 Developer and IT Pro Training

http://blogs.technet.com/b/miketag_sharepoint_blog/archive/2013/01/23/office-365-developer-and-it-pro-training.aspx

#7 => Microsoft Virtual Academy – Developing Microsoft SharePoint Server 2013 Core Solutions Jump Start

http://www.microsoftvirtualacademy.com/training-courses/developing-microsoft-sharepoint-server-2013-core-solutions-jump-start#?fbid=xne90rRJ2Go

Enjoy!!!!

As most of SharePoint Professionals, I have been keeping list of handy URLs to navigate through my day to day tasks. Recently I have came across this blog article – Famous SharePoint URLs and Locations and felt like everyone has their own go-to list of URLs. This article has excellent list of famous SharePoint URLs but one thing I felt it was missing was whether they would work for Site Collection or Sub Site.

Here are the list of all the handy SharePoint URLs I have been keeping track of. Let me know if I have missed anything or any of your favorites.

• Site Collection URLs
  • Get the SharePoint Version -  {SiteCollectionURL}/_vti_pvt/Service.cnf
  • Get the Site Collection User Information List – <SiteCollectionURL>/_catalogs/users/detail.aspx  or <SiteCollectionURL>/_catalogs/users/simple.aspx
• Site URLs
  • Login with different user – {SiteUrl}/_layouts/closeConnection.aspx?loginasanotheruser=true
  • Get the Site Security All People List – {SiteUrl}/_layouts/people.aspx?MembershipGroupId=0
  • SharePoint Foundation – Site Summary Analytics – {SiteUrl}/_layouts/usage.aspx
  • SharePoint Foundation – Site Usage Analytics – Monthly Summary Reports – {SiteUrl}/_layouts/usagedetails.aspx
  • Set up missing default groups post site creation – {SiteUrl}/_layouts/permsetup.aspx
  • Taxonomy Hidden List – {SiteUrl}/Lists/TaxonomyHiddenList/AllItems.aspx
• Page URLs
  • Get the Web Part Maintenance Page – <WebPart ASPX Page>?Contents=1
  • Open the Web Part Page in Edit Mode – <WebPart ASPX Page>?ToolPaneView=2
• Publishing Site URLs
  • Get the Publishing Site Manager – {SiteUrl}/_layouts/sitemanager.aspx

References

• Famous SharePoint URLs & Locations – http://blogs.msdn.com/b/how24/archive/2013/05/23/famous-sharepoint-urls-amp-locations.aspx

Hope it helps

I have been using SPUtility ever since I have started building Custom SharePoint Applications but once in a while I come across SPUtility functions to solve problems which I normally either find myself in the corner or scratch my head to solve  issues.

With this article, I will try to outline some of my favorite SPUtility functions and which scenarios you may be able to use.

SendEmail - Sends an email to specified email address

Sending email from ASP.NET application requires understanding of location of SMTP Server and Ports. Since SharePoint email & SMTP information is configured as Outgoing Email in Central Administration. SPUtility has outstanding function which would use Current Site Collection context to send an email using Outgoing Email configuration.

private void SendEmailResults(string toAddress, string subject, string body)
{
SPUtility.SendEmail(SPContext.Current.Site.RootWeb, false, false, toAddress, subject, body);
}

MapToIcon - Returns the Icon URL for any list item (e.g. document, pages, or list item)

There are times you may need to retrieve Icon for current document or list URL. e.g. while building Custom Search applications, search API won’t return Icon URL as managed property. There is DocIcon crawled property but it would return nothing but extensions of the documents. Thanks to SPUtility MapToIcon function, we have solution which would return IconURL as long as you have access to the current document or list item URL. Thanks to I LIKE SHAREPOINT for this wonderful tip.

private string getIconUrl(string itemUrl)
{
return "/_layouts/images/" + SPUtility.MapToIcon(SPContext.Current.Web, itemUrl, string.Empty, IconSize.Size16);
}

GetLocalizedString - Get Localized String from Resource Files

While building multi-lingual applications, there are times where you would need to access localized label from the Resource Files stored in the 14-Hive Resource or IIS AppResources Folder. Additionally, if you ever deploy resource files in the subfolders (not at the root of Resources folder), SPUtility function works great even your Resource files are stored in sub folders.

private string getLocalizedString(string label)
{
return SPUtility.GetLocalizedString("$Resources:" + label.Trim(), @"Company.Webparts\ResourceFileName", Convert.ToUInt32(System.Globalization.CultureInfo.CurrentUICulture.LCID));
}

This post is password protected. You must visit the website and enter the password to continue reading.

Second part of this series is continuation of what I had started with first part to discuss end to end process of building SharePoint 2013 home lab. At the end of the first part, we were at the stage where all the parts were arrived from Newegg.com and ready to be assembled.

Although first part of this article was detailed & very descriptive of different decision making process, this part is more of step by step guide of assembling picture guide.

Allocate Specific Time and Space

First thing first, unpacking PC parts and assembling them together can take 2-4 hours depending on your experience. If you have never done it, plan to allocate whole day (I am not kidding) because every step will take to you through unknown land and it can be overwhelming at times. Please plan to view newegg’s computer building process along with this article to get familiar with process as much as possible.

Whenever you are ready, plan to move all the parts to space where you want to build the computer. Plan to have flat surface near power supply where you can assemble and test the computer box as you build it. Once you move to the space where you would PC, unpack all the boxes of computer parts as you walk through in next steps discussed in this article. Don’t unpack all the boxes at the same time. Carefully open all the boxes with warranty papers and instructions along with stickers you can have it on your computer case to show off.

Step 1 => Unpack Motherboard and Install CPU on Motherboard

Unpack Motherboard – ASUS Sabertooth X79 LGA 2011 and put Motherboard on the box on the flat surface. Please be very careful where you put the Motherboard. Don’t put it directly on the table or flat surface. Plan to put it on the box itself to avoid any scratches. Very tiny little damage will make whole PC building experience worthless.

Next step is to unpack CPU – Intel Core i7-3930K and carefully install CPU on the Motherboard. If you are unsure, how to install your version of CPU on your version of Motherboard, there would be 100s of free videos on Youtube available. Please ensure you ground yourself before moving to installation steps. Here is the excellent video of how to install LGA2011 Intel Core i7 CPU Installation on the motherboard. I have followed this video to ensure I was careful enough to proceed with CPU installation on the motherboard – http://www.youtube.com/watch?v=H3myiMEW3EM

Step 2=> Install Heat Sink and Fan on the Motherboard

After CPU is successfully mounted, next step is Install and configure Heat Sink and Fan on the motherboard – COOLER MASTER Hyper 212 EVO and Arctic Silver AS5-3.5G. Before you mount the fan, there are couple of ways to apply thermal compound paste. You can use either one-dot approach or . Even though this video installs water cooler heat sink, I have used same approach for Air Heat Sink as well with fan. Here is the another video on how to install Coolermaster Hyper 212 Evo Review & Installation on LGA 2011 Socket Motherboard- http://www.youtube.com/watch?v=H_QVWBm4CI4

Step 3 => Install RAM

Next step is to unpack DIMMs – CORSAIR Vengeance 64GB (8 x 8GB) and install RAM carefully on the motherboard. Ensure they are snapped correctly on the Motherboard.

Step 4 => Install GPU

Next step is to unpack GPU – MSI N640GT-MD1GD3 GeForce GT 640. Connect fan and snap it on the motherboard on PCI Express slot. Plan to watch MSI GT 640 V3 – Unboxing Video – http://www.youtube.com/watch?v=K2ax7WqhyPY

Additionally, ASUS Sabertooth X79 comes with 40mm fan. After watching Installation of the 40mm Fan on the ASUS Sabertooth X79 Motherboard LGA2011 video – http://www.youtube.com/watch?v=4Jm5mDPK5TU, I was ready to jump on the next fun steps.

Step 5 => Test the External Build – Connect Power Supply & Monitor

At this stage, core components of the Motherboard is installed and it’s ready to be transferred to the Computer Case for final packaging. It is one of the best practice to go through dry run and test the external build. One of the major benefits of this process is you can test the computer system and connections before going through final installation and packaging process. Unpack the Monitor and connect to the Motherboard. Unpack the power supply – Rosewill BRONZE Series RBR1000 and connect major components to the Motherboard. Some of the major connections are – 24-pin & 8-pin Main and CPU Supplemental Power Supply to Motherboard, Connect GPU PCI Power if needed, and connect Power Supply to main power extension.

Many motherboards comes with small speaker which you can use to ensure system is configured successfully while turning on the power. Install the speaker and start the power supply. Depending on the motherboard, you may have On/Off switch or two pins which required to short by Philips minus Screw Driver. If successful, system will boot up and all the fan will be running upon turning on the motherboard switch. If speaker is connected, it will beep upon the system boot up. Computer monitor will display Intel Processor and Installed RAM information. If speaker turned on with multiple bips and computer display CPU fan error, please ignore this message at this time. You can resolve this issue by changing the bios setting afterwards.

Step 6 => Internal Build – Moving Motherboard into Case & Final Installation of Motherboard, Power Supply, and HDD, SDD, and DVD Optical Drive

If everything is successful, it is time to disconnect Power Supply and Monitor from previous steps and Motherboard would be ready to be transferred to the Computer Case. Unpack Computer Case – Rosewill BLACKHAWK Gaming ATX Mid Tower.

Open the case and remove the side panel from where you have full opening to transfer all the computer components into the case. Here are the major steps I have followed:

• First step is install the Power Supply with Fan facing bottom or outside of Computer Case. Mount the PSU with the screw comes from the PSU manufacturer.
• Next major step would be installing Optical Drive (DVD Player). Open the front panel and align optical hard drive with case. I was fortunate enough to have snap slots. I had to just slip in the Optical Drive in the slot and it would snap & lock without screw.
• Next steps are mounting HDD and SSD. Depending on the case, you may need SSD Mounting kit or if you are lucky enough like me, you can easily slip in both HDD and SDD to slots with screw less design. My case had mounting slots available for both 3.5” HDD and 2.5” SSD.
• It’s time to transfer Motherboard to the Case. Unmount the GPU (which may affect your ability to install all the screws in the motherboard) from the Motherboard and transfer Motherboard carefully to the Case. Align your motherboard on the case and half-tighten screws from opposite end until all screws of Motherboard are on the case before fully tightening up.

Step 7 => Cable Management & Connecting all components

One of the most challenging and creative part of computer building process is how to connect all the components with power supply and motherboard and how to still keep it beautiful without messing up all the wires look like Jungle. Plan to have Motherboard, PSU, and Case Guides handy if you need and how to move/tighten wires as needed.

For my configuration, here is high level steps I had followed.

• From Front Panel, Connected 2 USBs, 1 Audio (Mic/HeadPhone – HD AC 97 cable), Power Switch, USB 3.0, and SATA Interface
• Power Cables – SATA Cables to HDD, SSD, and Optical Drive, 24-Pin main power supply connector to Mother board, 8-Pin CPU Connector to motherboard
• Data Cables from Motherboard – SATA  6.0 to HDD, SATA 3.0 to SSD, and SATA 6.0 to Optical Drive
• Connect Motherboard Fan with 3-Pin connector on motherboard
• Connect 4 Case Fans with Power Supply with Molex Cables (I was lucky enough to shipped with Power Supply, Please plan to buy extra if your Power Supply Manufacturer doesn’t provide them)
• Once all the cables are connected, final step is carefully reinstall the GPU on the motherboard

And, here is complete system!!!

Step 8 => Test the Internal Build – Connect Power Supply & Monitor

It’s time to test the Internal Build. Connect the PC with Monitor and Turn on the Power Supply to ensure system boots up and display same message as External Build.

Step 9 => Complete the Installation – Cover up the Case

This is it. It’s done. Final step is put the panels of computer case back and connect all the peripherals like keyboard, mouse, monitor and it’s ready for OS Installation.

Step 10 => Configure and Optimize BIOS

Before you install Operating System, you must ensure BIOS is configured properly and uses all the resources efficiently. Please refer to motherboard guide for relevant information. Some of the major areas I had considered are follows:

• System Date and Time
• CPU Configuration – HT Support Enabled, Virtualization Enabled, 64-bit Supported
• Booting Device – DVD Drive
• Change AI RAM Settings to ensure RAM speed is properly configured as per installed RAM specs – XMP for RAM-1866 Mhz Speed
• Link to Fix Fan Error – Set CPU Fan Profile to Manual and Increase Minimum to 50%. This should run fan atleast 1000 RPM to ensure BIOS doesn’t complain – http://vip.asus.com/forum/view.aspx?board_id=1&model=P7P55D&id=20100106201817046&page=1&SLanguage=en-us

Step 11 => Installing Operating System – Windows 8 64-bit Edition

Here are the high level steps I had followed to install Windows 8 Operating system. Alternatively, you can watch newegg’s computer building process on YouTube. Last step of this process is no-brainer but I wanted to document here for completeness of the article. Just install the operating system and get ready for Hyper-V installation as discussed here and you are on your way to create and host SharePoint Hyper-V VMs.

• First step of the process is Installing Operating System. Download Windows 8 ISO file and make a bootable DVD for Windows 8. Here are the steps I had followed to create Windows 8 Bootable DVD in 5 Minutes using Imgburn Software – http://www.youtube.com/watch?v=BxIZa25bVXI. In my case, I wanted to ensure I install OS in 2 TB HDD. First steps I performed was to unplug SSD and partitioned HDD (C drive with 500 GB and D drive with 1.5 TB) during the Windows 8 Installation by booting up install disc from DVD drive to avoid accidental installation on SSD. This installs Windows 8 with default drivers comes with operating system on HDD.
• Next step is to plug the SSD back and initialize the hard disk from computer management -> storage -> disk management screen. Format the SSD and configure drive volume for SSD. In my case, drive volume name was “VMs”.
• Next major step the process is to install all the latest drivers required by the PC. Keep in mind that Computer Case, PSU, CPU, RAM, Storage/SSD/HDD, and optical drive doesn’t require any drivers. Most of the drivers are required for Motherboard (Chipset, Network/LAN, USB, and Audio) and Video Card drivers. My next step was to download latest drivers from Motherboard manufacturer (ASUS with Motherboard Model Number) and Video card manufacturer (Nvidio or MSI) for Windows 8, 64-bit operating system. There are couple of options to accomplish this task.
  • First option is download drivers from computer with internet connection by visiting manufacturer site (preferred approach because of latest drivers for specific OS) and copy to the thumb drive or optical drive and install on the PC.
  • Second option is use manufacturer provided CDs from Motherboard or GPU Vendor. In this case, install LAN driver from Motherboard CD, Connect NIC to the internet Modem, download latest drivers & install directly from the computer.
  • I had followed first option and installed drivers in following order – Chipset, LAN, USB, Audio, Video, and SATA.
• One important step I normally follow at this stage is backup all the downloaded drivers along with Windows 8 bootable disk to avoid these steps again in future in case of reinstalling the Windows 8 operating system on the PC.
• Next major step is Activate Windows 8 and perform Windows Update. Go to the Control Panel and run & rerun Windows Update until there are no more updates. Additionally, check the Device Manager to ensure all the drivers are installed.
• Plan to install Anti-Virus at this stage of the installation. Microsoft has multiple free version of bare-minimum Anti-Virus software for different client OS – Microsoft Security Essentials for Windows 7 and Windows Defender in Windows 8. Alternatively, plan to have more robust Anti-Virus subscribed software.
• Plan to install any software requires like Microsoft Office
• Test and validate all the major vital programs on the PC including Internet connection, Audio, Video, and other settings.
• It is important to backup image of final state of the PC before using it for SharePoint labs. This would allow you to restore the image without re-performing all the steps discussed prior to this step. Use Windows 7 File Recovering option from Control Panel to backup & restore the image. Plan to save backed up image on different drive than default C drive.

Resources:

• Installation of the 40mm Fan on the ASUS Sabertooth X79 Motherboard LGA2011 – http://www.youtube.com/watch?v=4Jm5mDPK5TU
• LGA2011 Intel Core i7 CPU Installation Guide H100 Water Cooler Linus Tech Tips – http://www.youtube.com/watch?v=H3myiMEW3EM
• Coolermaster Hyper 212 Evo Review & Installation on LGA 2011 Socket Motherboard- http://www.youtube.com/watch?v=H_QVWBm4CI4
• MSI GT 640 V3 – Unboxing Video – http://www.youtube.com/watch?v=K2ax7WqhyPY
• How To Create Windows 8 Bootable DVD in 5 Minutes – http://www.youtube.com/watch?v=BxIZa25bVXI
• Asus SABERTOOTH X79 Drivers fro 64bit Windows 8 – http://support.asus.com/download.aspx?SLanguage=en&m=SABERTOOTH%20X79&p=1&s=42&os=36&hashedid=RPOIwd0pKLoXnbFV
• Link to Fix Fan Error – Set CPU Fan Profile to Manual and Increase Minimum to 50%. this should run fan atleast 1000 RPM to ensure BIOS doesn’t complain – http://vip.asus.com/forum/view.aspx?board_id=1&model=P7P55D&id=20100106201817046&page=1&SLanguage=en-us
• Windows Defender and Microsoft Security Essentials: Which one do I need? – http://blogs.msdn.com/b/securitytipstalk/archive/2013/11/14/windows-defender-and-microsoft-security-essentials-which-one-do-i-need.aspx

Please note this code snippet has been tested on SharePoint 2010 using Visual Studio 2012. It should work fine with SharePoint 2013.

If you are working on multi-lingual or support multiple languages in your SharePoint application, there might be at times, you may need to access localized version of SharePoint User Profile Properties.

As many of you may aware, SharePoint User Profile system allows an administrator to specify localized version of labels while creating User Profile Properties. I may have spent day or two and one or more sleepless night to figure out how to localize User Profile Properties using OOB approach before stumbling into localized display name option in User Profile Administration page and where Microsoft may have stored these localized data. As it turned out, these localized labels are stored in the User Profile database unlike other SharePoint localized technologies like .NET Resource files stored in the Resources directory in SharePoint Root or App_GlobalResources in IIS virtual directory.

As demonstrated below, you can manage localized labels of User Profile Properties in the Central Administration. Please note that you must enable language settings for specified lanaguages in Central Administration site to add localized labels for User Profile properties.

As you can see, Localized values of User Profile Properties are stored in the User Profile Service Application’s Profile Database and PropertyListLoc Table.

Let me first point out, Accessing these localized display labels are not straightforward. Microsoft.Office.Server.UserProfiles namespace has CoreProperty class to represent the definition of basic User profile property which is common across all the User Profile Properties. This class represents most values required to define User Profile Properties including Name, Display Name, IsAlias, IsMultiValued, IsSearchable etc. To retrieve localized label of User Profile Properties, what we are interested is => CoreProperty class’s CoreProperty.DisplayNameLocalized collection.

As happens often with SharePoint, there are multiple ways to access User Profile Properties through Server Side Object Model and depending on requirements, one option might perform better than other. In this article, I will demonstrate two different ways to access Localized Label of User Profile properties.

Option 1 => Accessing User Profile Properties CoreProperty Class using CorePropertyManager, Much Better Performance

Once you get access to the UserProfilePropertyManager, you can directly access CorePropertyManager which would allow you to search all the Core Properties for specific User Profile Property and returns object representing CoreProperty class for specific user profile property. This approach allows you to avoid iterating through all the User Profile Properties and yields much better performance.

/// <summary>
/// Get the localized label of User Profile Properties
/// Option 1 => Get the specific property to find the label (much better performance)
/// </summary>
/// <param name="userProfilePropertyName"></param>
/// <returns></returns>
private string getLocalizedUserProfilePropertiesLabel_FasterOption(string userProfilePropertyName)
{
string localizedLabel = string.Empty;

//Get the handle of User Profile Service Application for current site (web application)
SPSite site = SPContext.Current.Site;
SPServiceContext context = SPServiceContext.GetContext(site);
UserProfileConfigManager upcm = new UserProfileConfigManager(context);

//Access the User Profile Property manager core properties
ProfilePropertyManager ppm = upcm.ProfilePropertyManager;
CorePropertyManager cpm = ppm.GetCoreProperties();

//Get the core property for user profile property and get the localized value
CoreProperty cp = cpm.GetPropertyByName(userProfilePropertyName);
if (cp != null)
{
localizedLabel = cp.DisplayNameLocalized[System.Globalization.CultureInfo.CurrentUICulture.LCID];
}
return localizedLabel;
}

Option 2 => Accessing User Profile Properties CoreProperty Class using ProfileSubtypePropertyManager, Slower Looping Through Option

Another option is loop through all the User Profile Properties in the ProfileSubtypePropertyManager. All the User Profile Properties are categorized by Profile Subtypes. This option forces you to loop through User Profile Properties collection to access specified User Profile Property class and object representing CoreProperty class for specific user profile property.

/// <summary>
/// Get the localized label of User Profile Properties
/// Option 2 => Loop through all the user profile properties to find the label..
/// </summary>
/// <param name="userProfilePropertyName"></param>
/// <returns></returns>
private string getLocalizedUserProfilePropertiesLabel_SlowerOption(string userProfilePropertyName)
{
string localizedLabel = string.Empty;

//Get the handle of User Profile Service Application for current site (web application)
SPSite site = SPContext.Current.Site;
SPServiceContext context = SPServiceContext.GetContext(site);
ProfileSubtypeManager psm = ProfileSubtypeManager.Get(context);
ProfileSubtype ps = psm.GetProfileSubtype(ProfileSubtypeManager.GetDefaultProfileName(ProfileType.User));

//Get the properties by section
ProfileSubtypePropertyManager pspm = ps.Properties;
foreach (ProfileSubtypeProperty profileSubtypeProperty in pspm.PropertiesWithSection)
{
if (profileSubtypeProperty.Name.Trim().ToUpper() == userProfilePropertyName.Trim().ToUpper())
{
//Get the core property for user profile property and get the localized value
CoreProperty coreproperty = profileSubtypeProperty.CoreProperty;
localizedLabel = coreproperty.DisplayNameLocalized[System.Globalization.CultureInfo.CurrentUICulture.LCID];
break;
}
}
return localizedLabel;
}

Enjoy!!!!

Please note that this article refers to only SharePoint 2010 and SharePoint 2013 On-premises. Both User Profile Service and User Profile Syronization Service are pre-configured in Office 365 and SharePoint Online tenants.

Here are the high level steps you need to follow to configure User Profile Service Application and User Profile Sync Service in your SharePoint 2010 Developer VM and On-Premises farm. This article also applies to SharePoint 2013 On-premises multi-server farm as well but it can vary slightly depending on your farm configuration especially how SP_Farm account is used during UPS provisioning process and how it required logon local permission without having local admin rights. Alternatively you can download step by step visual guide from my Sky Drive folder.

Important Notes

• Please plan to use this guidance only for Least-Privileged Developer VMs to learn how and what really happens under the hood when you configure User Profile Service and User Profile Sync Service. This is more of “How” article instead of “Why” article as a quick reference for me in future. For real-world multi-server farm building scenarios (or even hardcore Developer VMs), please plan to use automated PowerShell scripts like AutoSPInstaller.
• It is also important to note that although I have used SP_Farm account (Bad practice according to Spencer Harbar) to configure these services upon logging into the server, preferred approach would be using SP_Install account or any other Farm Admin Account with sufficient privileges to configure Services and Service Applications (AutoSPInstaller uses this approach) using PowerShell or Upon logging into the server or using Central Admin from remote server. Discussions about whether to use SP_Install or SP_Farm or any other Farm Administrator account is out of scope of this article. Since this is Single Server Developer VM, I have used SP_Farm account. In real world multi-server farm scenario, I would use SP_Install and AutoSPInstaller to configure these services.

Scenario

Configure User Profile Service Application and User Profile Synchronization Service with NTLM Authentication in Single-Server Least-Privileged Developer VM.

Planning and Pre-requisites:

• SharePoint Edition – SharePoint 2010 Standard or Enterprise, This feature is not available in SharePoint 2010 Foundation.
• Service Application Dependency – Managed Metadata Service and Managed Metadata Service Application – Although this is not required, it’s recommended to ensure all the multi-value user profile properties dependent on Managed Metadata term store works correctly. Please refer to this article for detailed step by step instructions.
• Logon Account to Configure Service Application – Access to SP_Install or any other Farm Administrator Account to login to the server for Service Configuration. I will be using SP_Farm account in this guide and SP_Farm is local admin on the machine. More specifically, SP_Farm account is domain admin on DC since I am configuring UPA and UPS on single server All-UP VM.
• User Profile Service Application Pool Account Identity – SP_ServiceApps, either already created or access to AD to provision new domain user for service account. Additionally, this account must be configured as Managed Account in the Central Admin. (this step is discussed in this article but it’s already created for Managed Metadata Service Application)
• User Profile Sync Service Account Identity – This must be SP_Farm account. After UPS is configured successfully, FIM windows services will be running under this account. SP_Farm account must be local admin during User Profile Sync Service provisioning process.
• User Profile Sync Connection Service Account Identity – SP_UPS, either already created or access to AD to provision new domain user account. This account doesn’t need to be configured as Managed Account in the Central Admin since UPS doesn’t support Managed Accounts. (this step is discussed in this article) and this account must have “Replicate Directory Changes” Permission on the AD Domain. This account will be used to perform the user profile sync and won’t be used to run any services or application pools

High Level Steps:

• Prepare the Service Accounts for the User Profile Service Application
• Configure the User Profile Service and User Profile Service Application
• Verify the User Profile Service Application Configuration
• Prepare the Service Accounts for the User Profile Sync Service and Add Sample Users to Import in SharePoint User Profile System
• Grant Replicate Directory Changes Permission to User Profile Sync Service Account in the Active Directory
• Start the User Profile Synchronization Service
• Verify User Profile Synchronization Service Started
• Import the User Profiles from the AD using Sync
• Verify the User Profiles Synchronization Process

Detailed Step by Step Directions:

Step 1 – Prepare the Service Accounts for the User Profile Service Application

For the Least Privileged Setup, create service application domain user account (e.g. Niks\sp_serviceapps). Majority of the service applications along with User Profile Service Application will run under this account identity. Ensure “User cannot change password” and “Password never expires” are checked.

Log in to the server as a farm account (e.g. Niks\sp_farm). Visit the Central Admin and register the “Niks\sp_serviceapps” as managed account. User Profile and other SharePoint Service Applications requires Service Application pool configured as Managed Account in the central administration.

Step 2 – Configure the User Profile Service and User Profile Service Application

Log in to the server as a Niks\sp_farm user. To setup User Profile Service Application properly, please ensure that User Profile Service is already activated on the farm. Please note that this step is NOT mandatory but much more preferred approach. If you don’t start User Profile Service prior to provisioning User Profile Service Application, it would start automatically for you.

Visit the Central Administration -> System Settings -> Manage services on server and verify that User Profile Service is started. This service gets started automatically. If it doesn’t start automatically, start the service manually.

To setup new User Profile Service application, Navigate the Central Administration -> Application Management -> Manage Service Applications to create the new User Profile Service Application.

Click New and select “User Profile Service Application”

On the new User Profile Service Application creation page, enter the service application name and Application Pool. Specify new Application Pool and use the “Niks\sp_serviceapps” as Application Pool identity.

Specify Profile Database Name, Server, and Authentication Info.

Specify Synchronization Database Name, Server, and Authentication Info.

Specify Social Tagging Database Name, Server, and Authentication Info.

Specify Profile Synchronization Server Instance where User Profile Service will be activated. If you already have My Site Host site collection, please specify My Site Host URL. In my case, I haven’t configured My Site Host site collection. Please note that this step checks for valid My Site Host URL.

Specify My Site Managed Path and My Site Naming Format.

Upon clicking OK, Both User Profile Service Application and User Profile Service Application Proxy is created.

Step 3 – Verify User Profile Service Application Configuration

After successful configuration, you can visit the User Profile Service Administration page by clicking the User Profile Service Application link. As you can notice stats on the right side of the screen, there are no user profiles. Also, notice that Profile Synchronization Settings are disabled.

For advanced SharePoint IT Pro related verification, please see Appendix A in downloadable document (link specified above) to understand what happens under the cover when you provision new User Profile service application.

In general, when SharePoint provisions new User Profile service application, it typically configures various components on the server:

• Central Administration
  • User Profile Service is started and User Profile Service Application and Proxy is created
  • Both SharePoint Farm Account and Default Web Application Pool Account are added with Full Control Permission on the User Profile Service Application Permissions Page
  • Associate User Profile Service Application with Content Web Application
  • Service Application Pool is configured for User Profile service on the Configure Service Accounts page.
• Active Directory
  • SharePoint Service Account is added to member of IIS_IUSERS and WSS_WPG group
• IIS
  • New Service Application Pool is created
  • New WCF Endpoint for User Profile Service Application is configured.
• SQL Server
  • New User Profile Service Application Databases – Profile DB, Social DB, and Sync DB are provisioned
  • SharePoint Service Account is added as Login account on the database Server
  • SharePoint Service  Account is configured with “db_owner” role on the all three User Profile Service Application databases
  • SharePoint Service Account is configured as WSS_Content_Application_Pools role in the SharePoint_Admin and SharePoint_Config database.
  • SharePoint Service Account doesn’t have any access to the content web application database

Step 4 – Prepare the Service Accounts for the User Profile Sync Service and Add Sample Users to Import in SharePoint User Profile System

To import the user profiles into the SharePoint environment, we need to configure the User Profile Sync Service. User Profile Sync service is a wrapper of provisioning of the Forefront Identity Manager (FIM) on the server.
To make sure My Sites, User Profiles, and People Search work more efficiently, login to the server using the “Niks\Administrator” and add More Users to the Active Directory using the Active Directory Services from the Start -> Administrative Tools -> Active Directory Users and Computers. Add User – Nik Patel. Copy the first user and create a small # of the users if you want.

Login to the server as “Niks\sp_farm” and make sure “Niks\sp_farm” account is the local admin when you provision the User Profile Sync Service on the Server. Add the “Niks\sp_ups” service account in the AD. This account will run the User Profile Sync Service and Perform the User Profile Sync using the FIM.

Step 5 – Grant Replicate Directory Changes Permission to User Profile Sync Service Account in the Active Directory

In order to be able to sync with AD, you need to ensure User Profile Sync account (e.g. “Niks\sp_ups”) have “Replicate Directory Changes” permission. You will need this account name and password when you create a connection. Please do not proceed without having an account with these rights. This AD right is required for both full and incremental sync. Open the Active Directory Users and Computers snap-in. On the View menu, click Advanced Features. Right click on the domain object – niks.local and choose Delegate Control. To verify the Replication Directory Changes permission on the “Niks\sp_ups” object, right click on the domain object – niks.local and choose properties. Select the Security Tab and verify that “Replicating Directory Changes” permission was added to “Niks\sp_ups” account. See the attached guide for details.

Step 6 – Start the User Profile Synchronization Service

To Configure the User Profiles Sync, Open the Central Administration. Click on Central Administration -> Manage Services on Server; provision the new User Profile Synchronization Service Application by clicking “Start”

Select user profile service application and specify the farm account – Niks\sp_farm password. Specify the User Profile Service Application instance if you have multiple User Profile Service Applications are running on farm. Additionally, please note warning in Red – If SharePoint Central Administration is deployed on this machine, you must recycle Internet Information Services after provisioning the User Profile Synchronization Service.

User Profile Synchronization Service status should be “Starting”. This step can take up to 10 minutes depending on the network and hardware configuration in the farm. Alternatively, you can monitor running jobs to see the progress of User Profile Sync service provisioning process by reviewing ProfileSynchronizationSetupJob on the Running Jobs page in Central Admin Monitoring section.

Refresh the Manage Services on the Server Page to see if User Profile Synchronization Service status is “Started”. It will take a while because this process provisions the FIM service on the server.

As specified earlier, run the IISRESET. It is required if FIM is installed on the same box as Central Admin.

Step 7 – Verify User Profile Synchronization Service Started

Run Services Snap-in through Administrative Services -> Services or type the services.msc on the Run command. On services snap-in, verify that “Forefront Identity Manager Synchronization Service” and “Forefront Identity Manager Service” windows services are started set to automatic, and running under “Niks\sp_farm” account. Do not start them manually if they are not running and troubleshoot the issue.

Since I have started User Profile Sync service on the Developer machine hosting both SQL Server, often FIM windows service may fail to start during VM startup process because of its dependency on SQL Server services. Although this is not necessary in Production environment because SQL server is hosted on different servers, plan to have FIM services started with “Automatic (Delayed)” to ensure FIM services able to start on the Developer machine hosting both SharePoint and SQL servers.

Check the folder %Programfiles%\Microsoft Office Servers \14.0\Synchronization Service\MaData to see if there are two subfolders \ILMMA and \MOSS-XYZ (where XYZ is the name of your user profile service application). These folders should be empty at this time.

Step 8 – Import the User Profiles from the AD using Sync

Open the Central Admin -> Application Management -> Manage Service Applications and Click on the “User Profile Service Application” to configure the AD Synchronization Connection. On the Manage Profile Service screen, click on the Manage User Profiles link to verify that there isn’t any profiles imported yet. On right side on stats, there are no profiles. You can also manage User Profiles, Audiences, User Profile Synchronization Connections, and My Sites from this page. Click on the “Configure Synchronization Connections” on the Manage Profile Service screen.

Click on the “Create New Connection”

Specify the User Profile Synchronization Connection Name, LDAP directory type – Active Directory, forest name – Niks.local, Authentication Provider – Windows for NTLM Authentication, and “Niks\sp_ups” as User to connect to the AD to sync the users. FIM will use this user to connect to the AD.

Click “Populate Containers”. Expand “Niks” domain, select “Users” and click Ok. Do not select all.

System may take while to create the AD synchronization connection. Once created, profile service should show new AD connection. If FIM services are not started, you may get error specifying MA Folder can’t be created while creating AD Sync Connection.

Based on the numbers of users and groups in your system, it may take much longer to sync users and groups. On the Manage Profile Services page, click on the “Configure Synchronization Settings” link. Select “Users Only”, Use SharePoint User Profile Sync and click Ok.

Now, you can synchronize the AD users to the SharePoint using “Start Profile Synchronization” link from the Manage Profile Service page. Select “Start Full Synchronization” and click Ok.

If you refresh the Manage Profile Service page, you will see the Profile Import Status may show “Synchronizing”.

Profile Synchronization Process can take long time depending on number of user profiles needs to be imported along with network and hardware configuration. Alternatively, to monitor the Profile Synchronization, Please visit the C:\Program Files\Microsoft Office Servers\14.0\Synchronization Service\UIShell and Click on the miisclient.exe. Clicking miisclient.exe will open the FIM/Metaverse Manager which is used by the SharePoint Sync Service to import/export/sync the AD profiles with the SharePoint. Please review attached guide for more details. Refresh the User Profile Admin Page after several minutes (approx. 10 minutes). Once Profile Import Status is Idle, User Profile Synchronization is completed. On the right side of the screen, you can see the number of profiles imported and other stats. In this case, there are 11 User Profiles are imported.

Step 9 – Verify the User Profiles Synchronization Process

On Manage User Profiles link, you should find all the profiles are imported.

Difference between My All-UP VM Scenario and Production Multi-Server Farm Scenario

This article provides detailed instructions to configure UPA and UPS in All-Up SharePoint 2010 VM with DC, SharePoint, and SQL hosted on single server farm. In production environment, most likely you would be running DC and SQL on different servers other than SharePoint 2010 servers. There are couple of major differences how I would deploy UPS in multi-server production farm scenario which can differ than this article.

• Logon Account – In this article, I have logged into the server using SP_Farm account which is local admin on the box to configure UPA and UPS service. In production environment, you would be logging in as SP_Install account to configure UPA and UPS service. I would use AutoSPInstaller to configure UPA service and manually follow steps from this article to configure UPS by logging in as SP_Install account on the UPS server.
• SP_Farm Account Permissions – In this article, I have logged into the server using SP_Farm account which is local admin on the box to configure UPA and UPS service. In Production, SP_Farm account needs to be local admin only during UPS provisioning process. You can remove SP_Farm from local admin after UPS is provisioned. Additionally, SP_Farm account requires to have “Allow log on locally” right on the machine running the UPS service, which can be granted via Group Policy or Local Security Policy on the UPS Machine. If SP_Farm doesn’t have “Allow log on locally” right on the farm without SP_Farm being local admin on the box, FIM windows services may not start properly.
• FIM and SQL Issues – Since I have provisioned UPS on the same server as SQL Server in my All-UP VM, I needed to make sure FIM windows services are started with “Automatic (Delayed)” status to ensure it starts after SQL Server windows services. In Production environment, most likely SQL Server would be running on different server than UPS service server. You don’t need to worry about changing FIM windows services status from “Automatic” to “Automatic (Delayed)” in the production environment with SQL and FIM windows services running on different servers.

Master References:

• Rational Guide to implementing SharePoint Server 2010 User Profile Synchronization – http://www.harbar.net/articles/sp2010ups.aspx
• “Stuck on Starting”: Common Issues with SharePoint Server 2010 User Profile Synchronization – http://www.harbar.net/articles/sp2010ups2.aspx
• TechNet – Configure User Profile Synchronization – http://technet.microsoft.com/en-us/library/ee721049%28v=office.14%29.aspx
• Test Lab Guide: Demonstrate profile synchronization for SharePoint Server 2013 – http://www.microsoft.com/en-us/download/details.aspx?id=38783

That’s it as far as Step by Step details of configuring User Profile Service Application and User Profile Sync Service. Don’t forget to check Harbar’s guides to understand “Why”. They are The best resources on the internet as far as User Profile Sync service for SharePoint 2010 and SharePoint 2013.

Note: Download Code discussed in this article from Here.

SharePoint 2010 introduced new ways to query Enterprise Search data using KeywordQuery API. For developers, there are two options available for custom Search applications – FullTextSQLQuery (Executes Search SQL Syntax queries) and KeywordQuery (Supports querying both Enterprise Search Index for SharePoint 2010 Standard and Enterprise product and FAST Index for FAST Server for SharePoint 2010 product) APIs.

KeywordQuery API is preferred approach to query data from SharePoint 2010 Enterprise Search engine. Last October I had demonstrated how to use KeyWordQuery API to build Custom Portal Navigation SharePoint 2010 Web Part using Search API. Additionally, Please note that – with both FullTextSQLQuery and FASTQuery are deprecated or obsolate in SharePoint 2013, Microsoft has strengthed their investment in KeywordQuery API in SharePoint 2013.

Pre-requisites

• You must configure Search Service Application on the Farm
• You must run Full Crawl to create crawled properties for any custom SharePoint list or document library metadata
• Create Managed Search Properties for any Custom Managed list or document library metadata and Run Full Crawl again

Solution

In this step by step guide, we will build Portal Navigation Web Part which would render  Site Collections Navigation by retrieving Site Collection data from Search Index.

Here are the high levels steps required for developers to build Custom Search Applications  using KeywordQuery API.

Step 1 – Create Visual Studio 2010 Solution based on SharePoint 2010 Empty Project Template

Step 2 – Add SharePoint Project Item for Search Application UI & Logic
Depends on what kind of business requirements, plan to add either Custom Web Part (Standard Web Part or Visual Web Part) or Custom Page (Custom Application Page or Site Page).

In our example, we will add Custom Visual Web Part for Portal Navigation Application.

Step 3 – Reference two Search DLLs from the ISAPI directory

Plan to reference Search DLLs from the Visual Studio Project, Search DLLs are available at SharePoint Root in ISAPI folder

• C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI\Microsoft.Office.Server.dll
• C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI\Microsoft.Office.Server.Search.dll

Step 4 – Import Search Namespaces in the code components

Next step to import all the Namespaces required to instantiate and run KeyWordQuery object.

using Microsoft.Office.Server;
 using Microsoft.Office.Server.Search;
 using Microsoft.Office.Server.Search.Query; [KeywordQuery Class defined in this namespace]
 using Microsoft.Office.Server.Search.Administration;

Step 5 – Instantiate the KeywordQuery object
You have two options, pass either current site collection (Classic MOSS 2007 way), or pass the Search Query and Site Settings Service proxy (New in SP2010) and instantiate KeywordQuery object

Option 1 =>

SearchQueryAndSiteSettingsServiceProxy settingsProxy = SPFarm.Local.ServiceProxies.GetValue <SearchQueryAndSiteSettingsServiceProxy>();
 SearchServiceApplicationProxy searchProxy = settingsProxy.ApplicationProxies.GetValue <SearchServiceApplicationProxy>("Search Service Application");
 KeywordQuery q = new KeywordQuery(searchProxy);

Option 2 (Preferred Method) =>

SearchServiceApplicationProxy proxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.GetProxy(SPServiceContext.GetContext(SPContext.Current.Site));
 KeywordQuery q= new KeywordQuery(proxy);

Step 6 – Set KeywordQuery Object Properties

Next step is to instantiate the KeyWordQuery Object and set the required properties. You can reference available properties on MSDN but here are the major properties you should be aware of.

• QueryText – This is keyword query text (e.g. LastName:”Patel”)
• ResultTypes – Search result types (default is Relevant Results)
• HiddenContraints – Allows you to filter by search scope
• RowLimit – By default, it returns only 50 records, Se this property required to be set to increase number
• RowsPerPage – Paging
• IgnoreAllNoiseQuery
• TrimDuplicates – Trims duplicate results
• EnableNicknames
• EnablePhonetic
• EnableStemming
• EnableSpellCheck
• EnableFQL (Allows you to run FAST Query lanaguage syntax on KeyWordQuery API on FAST Server for SharePoint 2010 server)

In our example, we will set the query to “ContentClass:STS_SITE”, which would return all the Site Collections in the specified scope sorted by Last Modified Date.

KeywordQuery keywordQuery = new KeywordQuery(proxy);
keywordQuery.ResultsProvider = SearchProvider.Default;
keywordQuery.ResultTypes = ResultType.RelevantResults;
keywordQuery.TrimDuplicates = true;
keywordQuery.RowLimit = 10;
keywordQuery.QueryText = "ContentClass:STS_SITE";
keywordQuery.SortList.Add("LastModifiedTime", Microsoft.Office.Server.Search.Query.SortDirection.Descending);
keywordQuery.HiddenConstraints = string.Concat("scope:\"", "All Sites", "\"");

Step 7 – Use SelectProperties string collection to Specify which Properties to Query in ResultSet

Use SelectProperties string collection to return custom managed properties
Out of the box, KeywordQuery API will return only default search properties such as Title, Author etc. To return managed properties, use SelectProperties string collection.

keywordQuery.SelectProperties.Add("Title");
keywordQuery.SelectProperties.Add("Path");

Step 8 – Execute KeywordQuery Object Execute() method and return ResultTableCollection which can be loaded in ADO.NET DataTable Object

ResultTableCollection results = keywordQuery.Execute();
ResultTable result = results[ResultType.RelevantResults];
DataTable dt = new DataTable();
dt.Load(result, LoadOption.OverwriteChanges);

Step 9 – Bind ADO.NET DataTable Object data into ASP.NET or SharePoint Controls for UI

blSites.DataSource = dt;
blSites.DataTextField = "Title";
blSites.DataValueField = "Path";
blSites.DataBind();

Final Source Code

You can download Visual Studio 2010 Solution from Sky Drive here. Optionally, you can copy and paste from here.

Markup Code

<asp:BulletedList ID="blSites" runat="server" DisplayMode="HyperLink" />

C# Code

using System;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Data;
using Microsoft.SharePoint;
using Microsoft.Office.Server;
using Microsoft.Office.Server.Search;
using Microsoft.Office.Server.Search.Query; //KeywordQuery Class defined in this namespace
using Microsoft.Office.Server.Search.Administration;

namespace SP2010VM.SearchSolutions.CrossSiteCollection_KQL_Farm
{
public partial class CrossSiteCollection_KQL_FarmUserControl : UserControl
{
protected void Page_Load(object sender, EventArgs e)
{
//Instatiate Search Service Application Proxy
SearchServiceApplicationProxy proxy =
(SearchServiceApplicationProxy)SearchServiceApplicationProxy.
GetProxy(SPServiceContext.GetContext(SPContext.Current.Site));

//Instatiate KeywordQuery object and set the query
KeywordQuery keywordQuery = new KeywordQuery(proxy);
keywordQuery.ResultsProvider = SearchProvider.Default;
keywordQuery.ResultTypes = ResultType.RelevantResults;
keywordQuery.TrimDuplicates = true;
keywordQuery.RowLimit = 10;
keywordQuery.QueryText = "ContentClass:STS_SITE";
keywordQuery.SortList.Add("LastModifiedTime", Microsoft.Office.Server.Search.Query.SortDirection.Descending);
keywordQuery.HiddenConstraints = string.Concat("scope:\"", "All Sites", "\"");

//Specify which keyword query properties needs to be returned
keywordQuery.SelectProperties.Add("Title");
keywordQuery.SelectProperties.Add("Path");

//Execute the KeywordQuery query and load the results in the table
ResultTableCollection results = keywordQuery.Execute();
ResultTable result = results[ResultType.RelevantResults];
DataTable dt = new DataTable();
dt.Load(result, LoadOption.OverwriteChanges);

//Bind the table to the bulleted list
blSites.DataSource = dt;
blSites.DataTextField = "Title";
blSites.DataValueField = "Path";
blSites.DataBind();
}
}
}

References

• Keyword Query Syntax Reference – http://msdn.microsoft.com/en-us/library/office/ee558911(v=office.14)
• Sorting search results in SharePoint 2013 (Still applicable to 2010) – http://msdn.microsoft.com/library/office/jj938031(v=office.15)
• How to: Use the MOSS Enterprise Search KeywordQuery class – http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/02/19/how-to-use-the-moss-enterprise-search-keywordquery-class.aspx
• How to: Use the SharePoint 2010 Enterprise Search KeywordQuery Class -http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/08/12/how-to-use-the-sharepoint-2010-enterprise-search-keywordquery-class.aspx