Latest Entries »

Two years back, I almost did not make it to Ignite.  As I was going over goals for 2015, word came out that in our group travel budgets had been cut by 60%!!  And then I was challenged to explain in writing how my attendance at Ignite would help our group’s internal customers; merely being valuable for my professional development was not enough.    Yet, dear readers, I was there.    Let me share some principles of invisible persuasion (wherein the target is not even much aware of the persuasion taking place) which brought me to Ignite, and which might help you next year.

Yes, there is a standard “Reasons to Attend Ignite” letter on the main website.  However, it is puny.  Remember, your manager may not care a fig about you, but definitely cares about your team’s customers, because your successful service to them affects how they look at him/her.  Leverage this, and start making plans for 2018!

  1. Map to your goals. Look at established goals for you for this year.  Identify specific sessions from Ignite which map to and support those goals.  In my case I was quite specific.  For example, I have helped run two SharePoint-based programs in the past couple of years.  For those programs, for those customers, and for those support needs, the Ignite sessions on the new features for SharePoint will be invaluable.  That’s just one simple example.   Be sure to mention non-break-out session features as well, such as test prep, vendors, etc.  Many of those cannot be downloaded.
  2. Map to your customers’ needs. Look at the accepted expectations from your internal/external customers and show how attendance at Ignite maps directly to those.  Things such as troubleshooting, migration, new technologies, etc., are usually great for this.
  3. Map to your customers’ questions. Got any customers asking you about the Cloud?  Hyper-V versus VMware?  Windows 10?  Tell your management that you need to attend Ignite to be able to better serve your/their customers, and make you/them look good with these customers, by being able to field queries like this.  Consider offering to present summaries to customers on key topics of interest to them.
  4. Map to key vendors. This may take more creativity.  Identify key vendors for projects/applications/responsibilities in your group. See what questions might be illuminated by working with the vendor’s A-team (certain to be on display at Ignite).    Or maybe for existing systems which have no major point of contact, you can take support or troubleshooting questions to the heavy-hitters likely to be on display at Ignite.  And, by the way, you can also use these discussions as superb networking opportunities for yourself.
  5. Emphasize more than sessions. This is obvious, but do more.  The breakouts are the absolute focus for most of us (that and the open bar receptions).  However, the unsympathetic manager can decide to give you an hour or two a week to watch videos as a cheaper alternative than sending you to attend.  So emphasize other aspects.  For instance, industry-specific sessions like the Healthcare IT, Higher Education IT, etc., as well as community sessions, are excellent for learning from your peers and representing your organization to them.
  6. Emphasize the value of the labs. Everyone, even managers, needs some understanding of major emerging technologies.  The hands-on labs are very efficient, focused, high-return options for accomplishing this.  They are focused, repeatable, and very well-supported, and cheaper than many multi-day training courses.  They help you to better serve your customers.
  7. Target influence. Think of your manager (or other decision-maker) and your manager’s peers.  Send back some links or powerpoints which are relevant for things they care about.  This helps them recognize the value of your attendance.  Particularly savvy is sending to your manager info/powerpoints/links which he/she can use to look good with upper management.
  8. Share the wealth. I’ve been asked to make a couple of group presentations when I get back to share relevant sessions from Ignite.  Opportunities like these further allow you to show everyone the value of your attendance.   Also, do not underestimate the value of passing out t-shirts, hats, vendor stuff, free booklets, cups, etc., to your friends and allies, _and_their_children_ (people may politely decline things you offer them, but they will seldom ever demur when offered something for their children).  Back in 2008 the brilliant marketing team for Server 2008 bought Red Bull by the case, rebranded it with a great Server 2008 design, and passed it out.  I brought back many of these, and saw them around the office as recently as last week!

These are a few ways to create support for your attendance.  Key lesson: demonstrate how your attendance will create value for your customers and thus your manager, and be sure to leverage your attendance for networking back home.  This will give you great competitive advantage even in a budget-constrained environment.




Every year some folks on the Channel9 forums or elsewhere ask about how to prepare for and get the most from their first technical conference.  Below I’d like to share some insights to help first-timers get full value from their investment in attendance.  They’re organized for easy reading.  Some apercus get mentioned every year (like comfortable shoes); others are more unique to my experience.  All should be useful.

  1. Distance-1. For most folks there is a huge amount of walking from one session to the next, almost enough to burn off all the hot dogs, ice cream, snacks, brewskies, and lunches.  Therefore, this is the wrong time to break in new shoes for any reason whatsoever.  It is the right time to wear the most comfortable ones you have.
  2. Distance-2. Remember that every session will be recorded and available for download within a couple of days.  Many conclusions follow from this, not least of which is that you don’t need to sprint from one end of the conference center to another.  All too often in the past, I would have to hoof it from Mark Russinovich’s fanboy-fest in the southwestern quadrant to Andy Malone’s security throw-down in the northeastern corner.  They were barely in the same time zone!   Your schedule can be based on convenience and propinquity rather than desperation.
  3. Dress Code. The good news is that there is not one.  Crisp office casual is okay, shorts and vendor t-shirts with goofy flashing lights and raffle tickets are just as good.  Unless you’re on the prowl for a job interview, wear whatever you want, as long as the shoes are comfy.  Conference t-shirts from Tech-Eds past will earn bragging rights.  I still have mine from 2001, though it would be a labor of Hercules to jam myself into it….   Every year someone suggests bringing a hoodie for the usually over-air-conditioned conference centers; I have never needed one yet.  There is no bad news.
  4. What to not bring. Don’t bother bringing more than a couple of days of shirts; there are usually dozens of vendors who will be giving them away.  It is glorious!
  5. What to bring. Moleskin for your feet.  Don’t forget power strips; power outlets are always at a premium, and these will earn you many friends.
  6. Luggage-1. Travel light and leave space for swag.  And even if you’re not the kind of guy or gal who likes to “go native” at tech conferences, think about the folks back home.  I have over a hundred t-shirts from various conferences (don’t ask me about my collection of X-Files, Ghost in the Shell, and WWII shirts!), but still bring back a couple of dozen every year.  They are a great way to make (or strengthen) connections at work.  And they have surprising uses; a, umm, close relative has absconded with a couple that she uses for dyeing her hair.
  7. Luggage-2. If you have too much loot, too many mugs and water bottles and hats and t-shirts and X-boxes and knickknacks for your luggage, consider UPSing some of the load back.  It’ll mean less to cart around at the airport, and you’ll likely save on the luggage charges.  Most conference centers have shipping centers.
  8. Umm, about downloading… Turn off your bit-torrent app on your tablet/notebook while at the conference. I always have mine on, seeding Grateful Dead and live music torrents.  This is quite frowned upon by the powers that be at Tech-Eds and probably at Ignite as well; it got my machine’s IP address blocked from Internet access.    Hotels are generally less fascist about this, but with a hotel full of geeks playing WOW, SWTOR, EVE, and Hello Kitty Online every night, good luck getting much bandwidth.
  9. Create value back home-1. Ask around at work to see what kinds of questions people have which they’d like to ask Microsoft.  Collect these, bring them, and hit up the dozens of MS product experts you will see at Ignite.  Get their business cards as well, so your colleagues can directly reach out to them.  The level of technical expertise for MS technologies at these conferences has no parallel in human history.
  10. Create value back home-2. Attend the sessions you want.  At the same time be on the lookout for sessions which might be valuable for folks back home.  Troubleshooting-oriented breakouts are a gold-standard example of this.  Send them links or forward powerpoints from sessions.  It creates added value from your attendance and strengthens your hand when you ask to attend next year.
  11. Create value back home-3. Do a quick survey to see what key vendors your colleagues are working with right now.  See what questions they have.  Then visit these vendors at the conference and get their insight.  I do this every year and it never fails to make life better back home.  Normally you get locked into working with one group of folks at a large vendors, whereas it might be that the expertise you need is with a different group.  Or vendors might send their A-team to get your account, and then start sending their C-team to you, in effect for training.  We’ve all seen it!!  However the A-team is always on display at conferences.  By the way, contacts such as these are a great reason to attend conferences; since they represent value which cannot be streamed or downloaded.
  12. Budgetary good news. You should not need to budget any money for food while at the conferences.  My manager grinned when I mentioned this.  Then I mentioned the likely taxi fares from my distant hotel to the conference center….  I clearly should have presented these facts in reverse order.  Learn from my error.
  13. Non-Breakout stuff. For me, and probably for you, the breakout sessions are the alpha and the omega for these conferences, yet they can all be downloaded.  So when grazing on my hillock of bacon at breakfast every morning, I check to see what other activities there are.   The exam prep chalk sessions?  Definitely worth checking out.  The community interest groups (like healthcare IT, higher education, women in technology) are fabulous ways to get new ideas and meet some new folks.
  14. Get out of yourself. At conferences I always sit at tables with strangers (a snap since I travel alone) and strike up conversations, asking folks what they think of the keynote, the sessions so far, etc.).  I have to admit, this is not how I met my wife or got my current job, but I have always enjoyed listening to people talking about what they found valuable.  In addition, if you are traveling with a group from your company, it would be comfortable but pointless to have all your meals with them every day.
  15. Time and Opportunity Management at the top. The A-game attendees arrive with a list of people they want to meet, have lunch with, or contact.  And obviously they have emailed them in advance to set something up.  It’s not too late to get started!  At the risk of sounding like a careerist, think of where you want to be in five years or so, and look at the gasp-worthy talent roster for Ignite presenters.  Of that crowded pack of pros, who could you benefit from contacting?  This networking is inevitably the least foregrounded but the highest ROI opportunity to come out of Ignite.
  16. Tier-2 Advice. There will be 14,999 people with the same backpack you have; tie a bandana around the handle to quickly identify yours.    Keep a journal and take pictures.  Photograph your luggage before departure in case it gets lost.  If you’re a guy, go easy on the drinkables; the lines for the men’s room are legendary.

Hope there’s something valuable here for everyone; I look forward to seeing the suggestions of others.   Next time I will share thoughts about how to make sure you can attend next year.

I’m attending solo, and am mercifully spared the logistical mischief of traveling with spouse or children.  Here are my best insights for preparation, in no special order.

–         Minimal clothes.  After all, hats and t-shirts will be cast my way all week long.  Don’t bother unless you’re planning on a job interview, or unless you have some awesome shirt like Ghost in the Shell or a Tech-Ed conference shirt from the ancient past, like 2002.

–         Reading material.  I always travel with a few books, for long flights, long waits, or early hangovers.  This year, Classical Chinese, Hastings’ Retribution, and Davies’ The Eerie Silence accompany me.

–         Airline food.  It’s not memorable.  No, really!  Even the pre-con lunches are better!  Pack a couple of sandwiches for the flight.

–         Power strip.  Bring one and make friends.

–         Your feet.  Take care of them.  It’s too late now to start getting in shape for the long hikes of Tech-Ed, so make sure you have your most comfortable shoes.  Not only that, get some moleskin, and pre-emptively apply it to protect your toes.

–         Get up early on Sunday (if you’re in town), and get over to the center as early as you can to register.  You avoid the madding crowds, and have a better chance of getting the style of backpack you want (if you are one to care about such things).  For non-pre-con attendees, your chance starts at 10:00 on Sunday.  Don’t dawdle; others won’t.

–         Don’t stress about taking notes at sessions.  You can get the entire powerpoint and download the video of the session in a day or so, which liberates you to spend your time listening and thinking, rather than trying to seize every syllable for long-term archiving.  This is an awesome feature.

–         Sessions and labs and vendors. Since you can download all the sessions, being there is not so crucial as we intuitively think.  I’m considering spending more time in the proctored labs and chatting with vendors.  That’s something I can’t just go back and get from Tech-Ed Online.    Yet, to be honest, it’s difficult to compare the pleasure of a Minasi session with a follow-the-numbers lab.   Perhaps the greatest resource at Tech-Ed is the strong geek contingent from Microsoft.  Some are mere managers, but here you have the chance to talk to the people who build the apps and OSs and developmental environments we all know and love.  I remember talking with the guy who wrote the code to allow us to pull AD objects from the recycle bin.  Extremely rewarding, and quite unique to this conference.

–         Think about those left behind.  Yeah, you lucked out and got support to attend.  But not everyone did.  Keep them in mind: solicit technical questions from them and make sure to get answers.  Bring back swag and t-shirts.  Write back some emails telling people how the conference is going.  Send notes about particularly good sessions to people who will appreciate them.

–         Keynotes.  Hey, these are streamed!  You can hang out or stay home and listen in!  On the other hand, if you’re turned on by techy power, and like close quarters, then the teeming acres of geeks at the keynote will welcome you.  I’ll be out in the hallway, with all the space in the world, notebook plugged in, alternating between Microsoft glitterati and YouTube.

I’ll be leaving on Sunday for the 2011 SharePoint Conference in Anaheim, Calif.  I’m especially interested in the chance to review and learn new insights from the masters about infrastructural aspects of SharePoint.  In addition, I’m planning on attending some of the show-and-tell sessions to learn about what other businesses have been doing with SharePoint, since the business stories can be strategically valuable, and are often much less predictable.    With technical conferences, the preparations are standard for most people, and best practices are regularly circulated.  Nonetheless, I have a few to share below, few of which that I have come across elsewhere.  Maybe there will be something useful for you as well.

Things to bring:

  1. Moleskin.  Bring it and use it; this is the best prophylactic against blisters, which are all too common when you walk a couple miles a day back and forth between sessions.  A must.
  2. Business cards.  Bring them and trade them.
  3. Bring a power strip.  Outlets are invariably at a premium in the conference rooms, and you will make many friends if you have one of these.  Until you leave and take it with you, that is.
  4. Vendors and birds-of-a-feathers.  Since all breakout sessions are recorded and available on the website soon after taking place, these should not be the exclusive focus of your planning.  Consider the other non-recorded events as well, such as birds-of-a-feather sessions.  In addition, there are some great vendors in the SharePoint workspace; it is worth spending some time getting to know some of them and their products.  Make a point of
    collecting business cards.
  5. Labs. Usually I ignore these, but since I can download all the breakouts, I plan to give some of these a try.
  6. Loot. Maybe, like me, you have enough t-shirts to last a lifetime.  But your co-workers likely do not.  Conference swag is worth getting not just for
    yourself, but for the folks back home.  At least half of the swag I bring back will be distributed.
  7. Download. The sessions are available for three months or so.    After that, it’s either nothing or your favorite file-sharing service.  So make
    sure to download as much as you can, as soon as you can.

While on the subject of SPC11, I’d like to make an observation on its website (http://www.mssharepointconference.,com).  The site has no forums or newsfeeds; the only novelty and populist participation comes from a trickle of tweets and bleets, most of which are simple bursts of zeal.  How unlike the Tech-Ed site, which now has been able to enter year-round availability.  Rather than re-invent the wheel, it would make sense for SPC to leverage the clear success of the Tech-Ed site.

One unique, innovative aspect of Tech-Ed this year was the Sunday night sessions on professional development.  The results were  uneven but intriguing, and augur well for the future.

They began half an hour or so after the standard pre-cons concluded (not enough time to get dinner, but enough time to nosh on cocktail  party food thoughtfully provided).  There were two parts:  a general session with two parts for IT pros and devs alike, and separate sessions for  these two groups.  Alas, I missed the dev meeting.

The general session began with Steven Frost, who speaks well, and clearly thrives in front of an audience.   His first subject was cautionary in nature: be careful what you do online, because it will be searched out by potential employers.  Some examples he gave were  revolting, but reflect the tenor of our times.  Even for IT pros, much of this was eye-opening.    After this shocker, he went on to  dispense more general, mundane career advice, and much of this was a series of bromides like something I would have expected from Dale Carnegie or Poor Richard’s Almanac:  “be on time”, “know your strengths and weaknesses”, and “expect that things will go well” are a few examples.

The guiding principle here is that you are a brand, and you must adjust your behavior on- and off-line to maximize the recognition and  value of that brand.   This is standard business thinking, really, but seldom discussed or embraced in IT, so Frost’s exploration gives it value for many people.  Naturally, the people who would have benefited most from it were not there.

The second speaker was Zeus (really!) Kerravala, who explored workplace IT trends, and their implications for the IT workforce.    This covered standard insights, such as the growing prevalence of smart devices, mobility, and consumer devices used for business purposes.  Nothing revolutionary here.  He did however also explore VDI as an emerging gamechanger which would permit workers to use their own devices without compromising IT needs.    This naturally has great potential for cost reduction, but the risks of data compromise and illegitimate access are resounding.  Moreover, this goes in the face of control-freak organizations and admins which like to control items as trivial and business irrelevant as personal screensavers and wallpaper.    Nonetheless, Kerravala argues these changes are here to stay, and he foresees a very near-term future when IT success will derive from its ability to deliver satisfactory user experience with social media, desktop virtualization,  collaboration, mobile applications, and appropriate levels of security and management.    There is much food for thought here, especially as we consider future directions for our own careers.

Personally, I’d say Frost was vastly the better presenter with more to say about career tactics, but Kerravala had more strategic career insight.

After this, IT pros and devs went their separate ways; I stayed with the former for a very loose panel discussion of career moves and  past choices.  The panel members were interesting, yet I kept hearing very standard, predictable advice, with very little that moved or surprised or  illuminated me.  In addition, since this was a panel discussion, no one had needed much advance prep, and this wing-it approach was very evident  throughout.  In the end it seemed to me that any half-dozen forty-something IT pros randomly recruited from the audience could have said much the same things.  For next year, I’d like something more substantive for this second half.

In retrospect, I was surprised that no one had made real suggestions for further study, such as Fortune, Harvard Business Review, soft skills classes, David D’Alessandro ‘s excellent Career Warfare (I’ll be writing about this soon), or even something as simple as industry trade publications.

The professional development sessions were worthwhile, especially since aggressive, risk-taking, strategically-oriented behavior is  not the first notion generally associated with folks in our line of work.  In principle such sessions are very valuable.  Next year’s would continue  this great start by highlighting different skills, bringing in different speakers, pointing to more offline resources, and adopting a more structured approach.

Just got back from watching David Chappel’s COS202, Introducing the Windows Azure Platform, His presentation delivers.  In 75 minutes, he gives a quick, comprehensive, strategic overview of the major elements and motivations in the Azure platform.  Clearly shows how the major elements are envisioned, and the new opportunities inherent in this.  In addition, he also spent time outlining the varieties of applications/systems which would thrive in a cloud environment, and also explored the business considerations. If anything I would like to have seen more on this, since many IT pros are oblivious to it.  Chappel himself is an exceptional presenter, and spoke with confidence and enthusiasmthroughout.  This is a time of change and a time of opportunity, and as he pointed out, a great time to be alive.

Definitely download and watch.  It will not help you study for the inevitable Azure Cert exam, but will help you understand the dimensions and the potentials of this major, disruptive new technology.

Tech-Ed-2008-Deploying MOSS with End-to-End Kerberos Authentication, Matt Munslow

In this 2008 Europe presentation, Munslow does a fair job of explaining material occasionally seemed complicated even to him.  Not brilliant, mind you, but fair.  He needs to work on posture; he tended to rock right and left like a metronome.  And the video reproduction is lamentable; the fuzzy quality of the close-ups makes the demos nearly unusable.  Nonetheless, there is some good insight about Kerberos and troubleshooting that do much to make this worth the effort.  This 400-level presentation from Europe in 2008 gets very serious very fast.  Note that this presentation focused on SharePoint 2007.    It retains value even in an SP2010 world due to its methodology and Kerberos explanations.

Munslow focuses on three logical blocks of material:  Concepts, Configuration, and Troubleshooting.  

First, Concepts.  How does Kerberos work, and how does this affect our MOSS implementations?  Munslow focuses on mechanisms within Kerberos which are germane to implementing SharePoint 2007.

1.  Service Principal Names.  Everything in a Kerberos realm (or AD domain) has a unique service principal name, and getting our system to work with it relies completely on correct name configuration.  Clients use these service principle names to identify and authenticate with services like SharePoint.  He gave an example of what these look like:

HOST/server (“server” means the NetBIOS name for the server) and HTTP/

“HOST” is one class of service principal name.  So is HTTP. 

2.  Keys.  Used to encrypt data; these are normally derived from hashed passwords.  There are two types- the long-term key, and the session key.  All domain objects have a long-term, master key, for authentication. 

3.  Tickets.  Data Encrypted with keys, used by clients to authenticate against services.

4.  Key Distribution Center.  Managed by AD and the Key Distribution Center (KDC).  This is comprised of a Database of objects and their related keys, an Authentication Service (initiates communication between client and AD) to provide ticket-granting-tickets which are used by clients to engage the third component of all this: the Ticket Granting Service.  This is the source of the tickets used to authenticate against SharePoint.   

5.  Sub-Protocols.  There are three exchanges related to this.  First is the Authentication Service Exchange, used by clients to get a ticket for tickets.  Then there is the Ticket-Granting Exchange, which is the issuance of a ticket to the client.  Finally we have the Client/Service Exchange, in which the client authenticates against SharePoint.    Each represents an opportunity for things to go awry, and Munslow wisely recommends understanding them for help in troubleshooting. 

Next, let’s review the authentication process and how it works.  Here we see there are four steps.  First, a client negotiates with an IIS server running SharePoint.  An anonymous request is denied, but the denial includes information on how the client can negotiate access.  Second, the client requests a ticket-granting ticket.  This is done by the client seeking such a ticket from an AD server.  This request relies on recognized client ID information and a “nonce” (number used once) value.  AD replies with a session key that allows the client to get other keys; this key is encrypted with the AD master key, serving to validate the request with other servers.  The nonce is also included, which serves to verify to the client that this ticket is from the server it originally contacted. And the client’s own master key can decrypt all this information.  Now the client has a ticket-granting ticket and a session key.  In Step Three, the client relies on these to get a ticket for access to SharePoint.  The client asks for a ticket for access to SharePoint.  The ticket-granting AD server gives this in a key encrypted with a session key.  Note that the AD does not have a copy of the session key.  It creates it and sends it, but does not retain it.  Thus the ticket-granting ticket is crucial.  This leads to a session key encrypted with a session key specific to the SharePoint service, and which will work, of course, only on the target server.  Now the client has what it needs to access the SharePoint server. 

Now, in Step Four, the client uses these credentials to contact the SharePoint server.  It sends the authenticator information again, with the service ticket provided by AD.  The service on the SharePoint server uses its own session key to unlock the ticket from the client, discovering all is proper.  Then it opens a session with the client. 

This is the theory and the concepts behind Kerberos with SharePoint; they should be familiar to anyone who has studied PKI.  Next, Munslow explains the Configuration process.    

Second, Configuration.

Munslow presents a fairly generic customer topology: 3 load-balanced WFEs (Web Front Ends), 3 application servers (one index and two query), and a backend cluster (2 active and 1 passive).  He sought to reproduce this in VMware, kerberizing it along the way.  First they did the web applications.    Kerberizing them required creating a domain service account, and using this account to create the web application; this is the account the application uses to represent itself.    Next, DNS entries were created, but not with CNAMEs.  Instead, it was necessary to rely on A records, which Munslow pointed out is non-intuitive.  In Step Three, they created SPNs (Service Principal Names).  They utilized a tool from the Windows 2003 Server Resource Kit, called setspn.exe.    This creates SPNs for the NetBIOS name of the service, and for the FQDN.     Fourth, for the web application, you configure it in SharePoint Central Administration to use Kerberos. 

Now, this was all in reference to a web app.  There can be complications with this, since a non-default port is used; during installation, a non-default port is used, or a random one is assigned.  Also, issues can arise from the fact that IE 6 (used on Windows XP) does not properly construct the SPN.  The non-default port would be left off.  Microsoft has a fix for IE that corrects this through a registry change.  Once done, things started to work.  Note that the use of W2K3S or W3K8S does not affect this, since it is, after all, a client issue. 

A demo followed.   Unfortunately the screen was too poorly recorded for this to be usable at all.  Even maximizing the screen was useless for rendering this intelligible. In light of the fact that several cameras were used to record this session, it’s quite surprising that the end result was such a disappointment.   Generally, though, working within the Central Administration to enable Kerberos was straightforward.  As straightforward as a topic like this can be, anyway.

Shared Services. 

After the web apps have been kerberized as show above, the next stage is to kerberize the shared services provider infrastructure, which is done by this clear but cumbersome stsadm command:

Stsadm.exe –o setsharedwebserviceauthn -negotiate

This does the same thing for the shared service provide which we did for the web apps.  However, Munslow discovered when this was done, search settings would break, with seemingly random error messages. 

Issue #1.  Munslow describes the torturous process of troubleshooting this completely undocumented problem.  He found enlightenment in the shared service provider infrastructure, specifically in the Office Server Web Services Web Application, which is created when the farm is created.  It runs with the application port called the Office Server Application Port and with the identity of Network Service.  Note that this service runs on non-default ports: 56737 and (secure) 56738.   Every time a shared services provider is created, it created a virtual directory in that web app with its own application port.  This runs as the identity of the account used to create the SSP, that is, the SSP service account.  The other important thing to note here is that it is the .NET client which uses the web application to communicate with other SharePoint servers.  This is how inter-server communication between SharePoint servers takes place. 

The thing is, the .NET client cannot use non-default ports to bind to the server. 

Issue #2.  Another problem occurs when you have more than one Shared Services Provider.  This is common when using Excel Services.  Here the difficulty occurs when kerberizing the services.  The system’s first impulse is to use the same name (albeit with a different user) for each, which is not allowed, and will lead to errors, due to having more than one SPN. 

Issue #3.  Yet another, and more widely encountered, issue pertains to the indexer, which cannot crawl Kerberos web applications on non-default ports.  To overcome this, an NTLM web application had to be created and extended as a Kerberos.  This the crawler would use the NTLM version, and users would use the Kerberos version.  Munslow’s group found that a crawler could crawl a Kerberos application as long as it was on a default port, such as 80 or 443.  Otherwise it would not work.  His work-around was to avoid using any non-default ports for any crawling web app.  No more elegant solution for this currently exists. 

For Munslow’s group, the interim solution for the SharePoint group was to discontinue the use of Kerberos.  This was disappointing but practical.    More than half a year after Munslow left this effort, a new format for the SPN was introduced which overcame this:  the MSSP.

The MSSP requires the creation of two SPNs for every server in the farm which is participating in shared services, which means one for the non-secure port 56737 and one for the secure port 56738.  This allows you to name the shared services provider; thus solving many of the issues Munslow mentioned earlier, since it gets rid of the problem of duplicates.    How to make this work?  First, the infrastructure update must be run on all SharePoint servers.  Then a manual registry tweak must be run, adding a DWORD entry, set equal to 1, in this key:


After this, do a reboot. 

Then register two SPNs for every server involved with shared services, as noted. 

Finally, go to the command prompt, and run

Stsadm.exe –o setsharedwebserviceauthen –negotiate

That should do the trick.  A demo followed; it was partially intelligible.  

Next, Delegation

To begin with, note that this is only needed if SharePoint is to pass credentials.  It is not necessary for simply logging in to machines.  For services not hosted by the SharePoint server, such as RSS, for instance, we are confronted with the so-called “single-hop rule”, which is meant to stop man-in-the-middle attacks.  The goal here is to allow SharePoint to represent a client in approaching these other services.  This can be done as follows:

 1.  Register the appropriate SPNs for the service.

2.  The server which does the impersonation (or, more charitably, representation) must be trusted for delegation.  This can be done (easily) in Active Directory Users and Computers through a constrained delegation.  Then, also within ADUC, add the service account of the RSS server. 

3.  The impersonating service also must be trusted for delegation.  This account too is trusted for constrained delegation with the RSS account.

With this done, the user will be able to be on a SharePoint server and use an RSS web part on a SharePoint page. 

And that’s how delegation works.

Third, Troubleshooting. 

For me, this is always one of the best and most rewarding portions of any presentation, and Munslow culled through his own customer experience to assemble these observations for responding to Kerberos-related problems.

  1. Duplicate SPNs.  This is the first place to look.  As noted above, this can be a source of great mischief. 
  2. CNAMES in DNS.   The only workaround is using A records.
  3. Use NETMON on the wire between the client and ASD and client and service, and look for errors.  Also check the performance of sub-protocols. 
  4. Check sub-protocols.  See above.
  5. User Event Viewer.  If you do not see Kerberos 540 events in Event Viewer, then there’s a problem. 
  6. Turn on Kerberos Logging for more detailed logging.


He also suggested some resources; perhaps the best of these were: for general information on the installation process for Kerberos authentication.  Very thorough and insightful.

Other resources were mainly MIT’s take on Kerberos and some RFCs.

So, what’s the value in his presentation?  Weakly recorded demos, weak body language in presentation, and some daunting material.  Not only that, the world is moving to SharePoint 2010.  True on all counts, yet despite all this, Munslow’s presentation merits some attention for two reasons:

1.  The whole world has not switched to SharePoint 2010 yet, and SharePoint 2007 will remain in use for many years.  Thus its Kerberos performance needs to be well-understood.

2.  The advice on troubleshooting is very good, and generally applicable for all versions of SharePoint.

When my former blogging platform inexplicably disabled line-skipping between paragraphs, it was time to go.  I moved to WordPress, so I greeted Jeff Siarto’s  book with excitement, because WordPress is powerful but occasionally non-intuitive.  He did not disappoint.

Head-First WordPress covers getting started with your blog and goes beyond the basics of adding entries, such as managing content and managing pages.   Siarto also covers cool things such as customizing your blog with video and audio, and adding contributors.  The book is ambitious, and early chapters assume you use a web server and a hosting service.  However even if you are but a simple blogger, there are good tips and help getting started, such as the adoption of themes and widgets.  I personally found some for the examples, such as adding textboxes, quite helpful, and have adopted them in my on blog,

I liked the presentation style.  Siarto offers a “multisensory learning experience”.  This means teaching strategies such as creative redundancy, a conversational prose style, anecdotes, and humor, to teach through entertainment.  Not easy with most subjects, but it succeeds here.   Since I have little interest in learning HTML or PHP, the many pointers and notes were ideal.  Ready-use CSS code is also included, a godsend to those with no interest in learning it just to blog. 

Good for high-school and college web publishing classes, and for anyone wanting to learn independently.     Oriented more towards ambitious web publishing efforts than garden-variety blogs.  Avaiable through O’Reilly, at

Normally, we don’t look to HR folks for insights about SharePoint.  And I’ll admit to feeling some inner resistance initially, and only went to this session because I was geeked out on earlier ones.  However, I was surprised that I discovered great value in this presentation from the 2009 SharePoint conference in Las Vegas:

  •  business motivations for technology change
  • good thoughts on implementing SharePoint
  • social changes impacting our corporate technological evolution


1.  The Story

Patricia Romeo explained the background: Deloitte HR had noticed many millennials leaving after less than a year, and discovered that younger people felt great difficulty in connecting with others, and never felt a sense of kinship around them.  This is especially true when compared with the vast (if shallow) networks they developed on FaceBook and similar sites.  As a result, Romeo started an effort to develop an internal equivalent which would permit people to connect in the course of their work, make a large organization feel smaller, and overcome deep silos. 

Her demonstration of the self-branding site was a typical, very straightforward SharePoint site, with business card information on the left (along with people she knows), “About Me” and interests/favorites in the middle, and resume/documents/publications in the right column.    The most curious elements to me were the many personalization ones, such as interests, sports teams, and even things to do in her city when visiting her office.    This reflects the millennial predisposition for mixing the professional and the personal in life.    In the case of D Street, a blog is also integrated; in the case of Deloitte, professional and personal topics are featured.  The end result has been 24,000 active profiles, 2000 blogs, and hundreds of visitors a day, and an application which is rivaled only by T&E. 


2.      The Technology

Rob Foster then came on explaining more of the infrastructure aspects of D Street.  In particular, he outlined the feeds from SAP that pre-populate “business card” components, and ways for users to incorporate and control feedback and guestbook inputs, delegations, etc.  Even Deloitte partners got involved in creating and editing their own profiles. 

The system was build to accommodate 50,000 users, ❤ second load times, consistent look and feel across profiles, and ways to monitor/moderate use and misuse.  Their legal and risk departments were surprised to find that few issues emerged from the D Street implementation; it seems that the maturity and professionalism of their work force pleasantly surprised them.

Scalability.  He found SharePoint scaled very well for D Street- they found that two WFEs for 50,000 users using blogs, personal sites, and collaboration sites.  Similarly, for the Deloitte user base they found that a single indexing and a single search/query server were enough.  One major (and surprising) weakness here was Foster’s failure to include specs on the machines used for this: RAM, proc type and count, etc.  Points off for this.  One customization in their architecture reflected the large amounts of information users were posting.  This necessitated multiple content databases.    Foster gave a logical application view of the architecture, but the material was too difficult to read (even with a freezeframeable video!) and too little explained to make much sense.  This is very regrettable, since the material seemed very promising.  In fact, for the techs in the audience (probably 90%, conservatively), this was some of the most fascinating content.  It deserved more. 

The deployment was incremental, in groups of almost 10,000 users.  The standard version of SharePoint 2007 was used, and all profiles were provisioned within seven days (!). 

Mercury tools (Load-Runner) were used for load-testing, and showed very high load-test times; this was addressed through a caching layer over the top of the web parts.   Keeping environments in sync for the load-testing was also a challenge.   

Lessons Learned.  Foster spoke to lessons learned from all this, post-rollout.  One, blogs should only be generated at user request, to avoid “dead air” in SharePoint and improve loading times.  Doing this reduced their indexing time from weeks to about three days.  Two, the site creation process is key to performance, stability, and maintenance.  Foster’s process was to create sites when accessed (i.e., on demand).    Also, the process no longer relies on site templates, leading to improvements both in site creation and subsequent performance.  Finally, they needed to improve the search to handle wild-card searches for people.    

Future Plans.  In the future, they plan to integrate D Street with the Deloitte internal portal.  Dashboards will allow them to personalize this for each employee; widgets can easily be added at will.    Social networking has also been added to make it easier to make connections throughout the organizations.   They also discussed the development of communities information, to create more levels for employees to network, and to even achieve business purposes on occasion. 

Audience questions started with the toughest: How does this add to productivity?  Romeo took this one and said that it was comparable to email.  No one can imagine not having email now, but what numbers could be provided to prove this?   

Conclusion:  Overall, this seemed like a fairly generic implementation of SharePoint 2010, and it’s not intuitive why this was rated a 300-level session.  But that’s not a slam; the presentation was noteworthy for its exploration of business imperatives at work in technology design and implementation, and especially for the way that larger social trends inform business technological change.

Even though this Tech-Ed 2008 presentation by Mike Watson, a Microsoft Technology Architect, is from 2008, there are many valid concepts here which will remain valuable far into the future.  Listening to this, it seemed to me that much of the HA/DR information here is fairly generic, with relatively little which was completely specific to SharePoint.  Nonetheless, Watson shows how these are excellent principles, and worth mastering, since they will not only help with SharePoint but SQL and other high-criticality systems as well.

1.      Measuring Availability:  “It’s all about the “Nines” 

Mike explains what various nines mean.  So, for example, 99% uptime means 87 hours of downtime permissible a year.  99.9% cuts this to just nine hours a year.  Not bad, most of us would agree.  But then, going to four nines, we see less than one hour allowed per year.  The famous fifth nine, 99.999% uptimes, means that every year a mere downtime of less than five minutes can be allowed. 

In practical terms, three nines is about all we can realistically achieve.   Here’s why.  Just five minutes in overages for maintenance can break five nines.  And a SQL cluster failover takes up between 5 and 20 minutes.  For patches, there may be four hours a month.  So, obsessing over five nines seems scarcely realistic and an unproductive diversion of resources.  Watson’s first task is to establish realistic expectations.

 2.     So, what are the most likely outages you will experience?

In Microsoft’s experience, 10% of downtime results from botched maintenance.  Some 20% came from web server capacity issues.  Another 30% was due to poorly maintained storage affecting SQL.    Hardware failures account for 20%, and everything else under the sun represents 20%.    Mike presented some best practices which have emerged from MS’s own experience with its own systems. 

 2.1       How to prevent botched maintenance?  This is an example of the small thing becoming a big hassle, and Watson’s suggestions are to avoid mistakes:

  •  Never take maintenance for granted or trivialize it.
  • Use a realistic test environment.
  • Minimize changes, or patch testing will become a full-time job.
  • Understand and own your system dependencies.

2.2       Automate standard issue responses.  Mike gave an example of ten lines of C# code which could automatically remove an ailing server from an NLB. 

2.3       Prevent web/app server capacity issues by always using at least three servers, planning for peak capacity, and considering the performance impact of background services.  And whenever possible, automate failover.    Avoid being stingy when provisioning web servers. 

2.4       Prevent SQL capacity issues through aggressive, generous growth projections.  Pregrow databases to a comfortable scale, say 100 gigs.  As soon as SQL reaches 75%, immediately start adding storage or scale.    Beyond this, optimize your performance by using 64-bit, and planning disk IO carefully; TechNet has excellent resources on this.  One example is his recommendation of 2 IOPs/gig set for temp, logging, and search.  For data, 1 IOP/gig. 

2.5       Specifically for SharePoint Online, he has this recommendation for server design: 32 gigs of RAM, 5 TB of storage, redundant power supplies, redundant fans, and two NICs, with two 2.33 GHz quad cores for the CPU.  These are really cheap, he says!  And that was in 2008!  Now you can probably get one with a Big Mac.

2.6       Preventing hardware failure is a crucial skill for all of us, and he has some specific ideas.  First, never use end-of-life or older machinery for production.  Second, since servers are cheap, buy redundant components.  Third, scale out with multiple servers to avoid single points of failure.  Fourth, for all OS and App drives, use RAID 1.  Fifth, and finally, follow vendor recommendations for maintenance and ensure the presence of redundant power through generators or batteries.   

2.7       Watson makes a very good point about dependencies.  Understand them, and arrange your SLAs so that your success does not become hostage to someone else.  “Trust but verify” warns our presenter.

3.       Design for Availability 

In principle, there’s no upper limit to the number of servers a farm can encompass.  That said, there are guidelines.  For example, web servers should start with three, to guarantee access and failover.  Multiple instances of search should be available, both for failover and speed of response.    Thus, for an organization with <50,000 users, Mike advocates for three web/query servers, and a single index/excel server.  For up to 75,000 users, he recommends four web/query servers, and two index/excel servers.  For up to 100,000 users, he proposes one more of each. 

In contrast, he presented a map of MSIT SharePoint.  This was vastly more complicated, with several web portals, each with its own NLBed web servers.  NLBed SQL servers went up to 8 in number, offering up to 6 terabytes of storage.  The largest unit is the EMEA SharePoint, which includes three servers in the Web/Excel/Query role, one in the Search Target role, one in the Index role, and 8 clustered (active/passive) SQL servers offering 6 TB of storage, 

In such large farms, database mirroring is a vital component of HA, and Mike covers it.  In this system, two (or more) databases are linked by encrypted channels, comprising a Principal and a Mirror.  They are in turn monitored by a “witness” server which might be running SQL as simple as SQL Express (though this one is not recommended).    This latter server monitors the linked ones, and when the Principal is offline, it designates the Mirror as the new Principal.  Note that this automatic failover is an option for the SQL servers, but not for the SharePoint farm servers.  SQL 2005 SP1+ can be used for this, Mike recommends Enterprise because it provides for multi-threaded re-do queues.  This allows for better scalability and speed for applying transactions in SQL mirroring. 

 Watson covered three types of database mirroring.

 –      High Protection: synchronous.  Failover is manual, and transaction writes are synchronized on both servers.  Here there is a low tolerance for latency and performance weakness. 

–      High Availability:  This is the same as High Protection, but with the addition of a Witness server which manages any failover.  This is also synchronous.

–      High Performance:  Writes are not synchronized on both servers; here there is an assumption that everything will be completed successfully on the mirror.  Unlike High Protection, the High Performance mode can be tolerant of latency and low bandwidth.  Unlike the first two, this mode is referred to as asynchronous.

The first two are a local strategy.  The third mode is more suited to remote relationships, as we might expect from the tolerance for latency. 

Several things should be noted here.  SharePoint is not mirroring-aware.  This means that in cases of failover, the SharePoint Farm Servers need to be manually notified, as by use of the command stsadm –o renameserver oldservername newservername.  However, this command is not recognized everywhere within the SharePoint stack.  This means that another technology is needed to make this work. 

A SQL Connection Alias can also make a difference here.  Thus on an application server, you can define an alternate name for a SQL node.   So on your application server you configure an app to use one server, you can configure an alias which points to another server.  Then on failover, the alias is updated to the second server.    Watson then presented a detailed comparison of failover clustering and SQL High Availability Mirroring.    Some key points were that SQL HA Mirroring protects against failed storage, can leverage cheap DAS storage, allows up to 1 ms latency, and accommodates easier implementation of patches and upgrades.    Clustering, on the other hand, is automatically self-correcting, has a simpler recovery model, claims no performance overhead, and represents a minimal operational burden.  More could be said, but Watson begged off, referring to “a long night last night”.  What? 

His next slide helped to make up for this, describing a “Really Good H/A Solution” of active/active principal/secondary sites.  In this case you have separate facilities, distant by no more than a millisecond of latency, and with LAN-like bandwidth, a gigabit.  Both sites boast the SharePoint web/apps server, with the SQL backend of a principal and a mirror and a SQL witness at the main site.    In this situation, SharePoint/App servers at both locations both work with the principal’s SQL, which in turn mirrors to the secondary’s SQL.  In the event that the primary SQL goes down, the SQL witness redirects traffic to the secondary, automatically exchanging their roles.  The main vulnerability of this arrangement is some catastrophe which engulfs both facilities, such as blackouts, volcanoes, floods, hurricanes, tornadoes, or military activity.    Watson towards the end showed a more long-distance arrangement designed to overcome such catastrophes, involving local mirroring for high availability, and log shipping across the country for DR. 

The presentation was not perfect.  For instance, it would have been worthwhile at this point to have a discussion of the business reasons for choosing one or the other HA/DR options, since in most organizations technical considerations alone should not drive the process.  And it can only be disheartening to hear your speaker confess to “a long night”.  Nonetheless, this presentation was a good survey of the basic issues, and is a good introduction to HA and DR for any version of SharePoint.