Imagine that you are the CEO of a company and you are determining what to do with all your heterogeneous data that is residing
in your company in all sorts of information silos. That is the problem a lot of organizations deal with nowadays. In this blog I will give you on a high level the possible solution to the problem under certain circumstances. This means that there are circumstances which could be different than the ones you are dealing with, but probably you will recognize some mutual elements which could lead to some good ideas in your specific situation. Furthermore, I will give a sketch of the hard and software available in this specific situation and that also could be different than the one you are dealing with. Again: probably you will see similar elements which could lead to great ideas in your situation. Final: it is always possible to create a more similar situation with the one described here by buying in new hard and software.
Let us first see what kind of situation we have in this case. I’m the CEO and I know that I have all kind of data residing in several possible information holders. My managers tell me that this situation costs us a lot of money because a huge pile of information of quite some important value for the company is difficult to access from everywhere. Every department in the organization has its own way of saving and categorizing the data it needs, while this information is also important for other departments. Customers must wait sometimes quite a long time to get some answers. Furthermore: some decisions are not made or badly made because people lack the proper information to make those decisions. Managers tell me that our speed of reaction to the market is dropping and the same goes for our sales numbers. There are more phenomena in the organizations which tells me as a CEO that getting the right information on the right time and with the right speed is the overall cause of a lot of problems we deal with at the moment.
From out strategic considerations I want my company to have the right information at the right time, at the right places and within the right applications. I want my company to be able to react to outside information with light speed and my customers must get answers fast, not now but yesterday. I want the information being accessible from anywhere in the organization and from out every application my employees work with. That’s the only thing I want, how difficult could that be? Well, that’s what we will discuss in this blog and we will not analyze all the possible situations out there, but we will choose one that should be quite common. Let us first see what all the places are where our information can be found.
First of all we see that we have licenses for Microsoft SQL Server and that several departments use that as the information holder. Second we see licenses for Microsoft Office throughout the organization. What is used most is Word, Excel and Outlook. Outlook is our main email client. We have intel based computers and Microsoft Windows is our main operating system. There are some other Unix and Linux machines where we can find some financial and recourse management information. Some of those machines have crucial data on it where our customers depend on. Furthermore there are some Oracle machines with Java as the main language. Also we have some Oracle databases and again: also on those boxes we find crucial information. Last but not least, we have SAP with a huge amount of crucial HR information. My analysts tell me that also crucial data can be found in the email stores. It happens that this scattered email information is crucial in a lot of decisions we have to make. Last but not least, people tell me that the way we save documents is a mess and those documents hold crucial information. Most documents have some sort of Microsoft format, but the documents on the Unix and Linux machines have other formats. There is Microsoft.NET C# and Java as programming languages. This is the situation and it sounds like a real mess to me. Let us look at the picture.
Because Office is our main application, it would be unwise to first invest in getting the people used to another kind of Office like OpenOffice for example. Besides this fact, I also like the commercial model with which we can do good business. Open source is great, but it lacks static versions, the proper helpdesk model and every open source programmer has its own set of tools to hack around. I don’t say this is bad, but I say it is something I do not want at this critical moment in my organization. I want to be able to blame someone if things get screwed up or if software doesn’t work. If you want a real static open source environment, my experience is that you pay big money eventually. At this moment I don’t have that money and still I want this problem solved.
So, I listen to my people advising me to use Sharepoint. I point them wisely to a blog about Sharepoint and Change management and my managers agree that we first have to deal with that. That means, following some sort of model to get our change management structures in place before we introduce Sharepoint. A great blog told us if we don’t do that, that rampage will follow. We don’t want that. The blog also told us that we must introduce Sharepoint top down and not bottom up. Do not give Sharepoint to our system administrators too early in the process or else we will end up with another mess and we already have a mess. For further reading on this one:
Right. We have our change management things in place. We communicated our vision and the vision was: “ We want all our information accessible any time and any place and we want to do that with Sharepoint.” This vision is conforming what other people tell us about a vision. It is short, to the point, crystal clear and people can remember it. Every layer in the organization can understand what this vision is about. See the above links to read about what we have to do after forming this vision. When we dive into the technical things on a conceptual level, we assume that the information given behind the above links is being done and that the first stage is properly in place. If you do the things described in the following paragraphs and you didn’t do something in the form of what the links told you to do before implementing Sharepoint, than you are an unwise CEO and probably one with lots of extra problems in the near future. Don’t let that happen.
Let us continue. We have implemented Sharepoint and we did it with the links about Change and Change Management. We did it top down, we have a guiding coalition, we have all our managers and leaders aligned and we are ready to Sharepoint rock and roll. Now we can give Sharepoint to our administrators after we bought in the proper knowledge about Sharepoint in the form of education or new people. Don’t start the rest of this blog if you did not do that. Let them be certified and let them read the proper blogs and rss channels before you let your people play with the essence of your organization: the information. Their goal is simple: get all the needed information into Sharepoint and make that information searchable and accessible for everyone and every application. It sounds daunting, it is in the basic quite simple.
Before you put the information into Sharepoint, first think about the structure in which you want to have it. That is, how many site collections do you want to have? For which departments must there be a site collection? How many webs are allowed in every site collection? Who is responsible for the overall architecture of Sharepoint? Who is responsible for the architecture per site collection? Are we going to use personal site collections or My sites? What kind of search will we use? FAST search, out of the box search, our own search mechanisms?
What kind of security will we use? Ntlm, kerebos or claims based and what else do we have? What kind of lists will we use? How many content types will we use and what is the maximum complexity level of these things? Are we using workflows? If not, will we start using workflows? For what will we use workflows and for what not? How do we index all our data? What is in Sharepoint and what in SQL Server? How do we do the proper backup? How do we keep Sharepoint fast? And there are many more questions. That is not what this blog is about, but you should have this properly organized before you begin with the rest of this blog. There are many nice places of information and books on this matter and future blogs at this place will dive into these things.
The main topic of this blog is how do we get the data in Sharepoint to all the other applications in my organization if I’m the CEO? It’s about the step after all the decisions of the above paragraphs. Sharepoint is bought, it is in place, it has all the data but that data must flow to all our other business applications. The first barrier we have solved. All the data is now in one place. How it got there is not described in this blog. That will be the topic of other blogs. What we will describe is a possible solution about how to get that information from out Sharepoint into other applications. A part of this solution could also be used to get the data into Sharepoint and when that is the case, I shall mention it. I’m talking about Sharepoint on premise. Future blogs will dive into Office365, that other awesome product of Microsoft rocking the world.
There are two object models of Sharepoint which can be used to get data out of it or into it. You have the Server Object model and the Client Object Model. The big difference is that the Server Object model can be used if you are on the box where Sharepoint itself is residing and the Client Object model can be used if you are on a box other than the one where Sharepoint is residing. The Object models themselves have minor detail differences, but the way you program against it is similar. These Object models could be used to get information out of Sharepoint but also to get information in it. I can imagine that these object models were used when aggregating all the scattered information into Sharepoint in the previous steps.
The client Object model is light weight en general, but for the Unix and Linux machines where Java is king, it could be made even more general. This can be done with another layer called WCF, Windows communication foundation or plain old Web services. The funny thing is that we can make a Sharepoint project in our Microsoft development environment wherein we define a WCF service talking to Sharepoint objects. It’s a service which will reside in the Sharepoint catacombs. In that same solution we could make another WCF project outside the Sharepoint project and this WCF project can use the Sharepoint specific WCF layer. We make a loosly coupled layer around Sharepoint which is not Sharepoint specific anymore. This non specific WCF layer can also be made accessible true plain old Web services and these are even more easy to use in a Java environment. Another huge advantage of this theory is the fact that we can make our own objects which can be made more aligned to our business objects or internal object model. This idea can also be used to get information into Sharepoint.
By doing this we have created a layer which can be used easily by the Java clients on the Unix and Linux machines but we also made a layer which can easily be used by our other office applications like Word, Excel and Outlook. As mentioned in the previous blog at this place, we saw that we can use the Office applications to create Office Business Application where you can access all kind of information from out your other applications. We did that in the past and we can do it still. We also could have used this without first putting Sharepoint in place.
You can make a proper Web service or WCF layer with standard interfaces which access all kind of information layers in your organization, like the Unix and Linux machines. The problem with that solution is that you have to make a huge business layer which hold all the business specific logic to deal with the data. And the information has to be aggregated somewhere and that means extra databases and extra tables, relations and so on and so forth.
That is just the reason why Sharepoint came to existence, among other reasons of course, to have a place where all our business information will reside and which is accessible throughout the organization with all kinds of business rules, workflows and what not. So, we first introduce Sharepoint, put all the information in it in smart ways and then we make that information programmatically accessible to all our other business applications and that is what we are doing now. So, we do not use the Business Connectivity Services while that is also a great tool to get the data aggregated into Sharepoint to be used or shown there or make it available to other applications through for example External Content Types. Instead we use another WCF and Webservices layer to talk to Sharepoint specific WCF objects to present the Sharepoint information to other application and not all of them are Microsoft based. With this technique we can make our own business objects and this also can be used to get information into Sharepoint.
An interesting article about integrating Sharepoint Office Business Applications with Business Connectivity Services and External Content Types can be found here:
We came a long way and we are almost there. The vision was “We want all our information accessible any time and any place and we want to do that with Sharepoint.”. First we’ve put the information into Sharepoint in a structured and smart way and following the blog about Change and Change management and that sort of things, do not underestimate that part. Then we made the structure accessible from out Sharepoint itself with all kinds of lists, content types, workflows and what not. After this step we created a Sharepoint specific WCF layer in Sharepoint projects which is talking to the objects of Sharepoint.
Because we have so many non Microsoft places where the information must also be accessible, we choosed not to use the server or client object model directly but we’ve put a generic WCF and Webservice layer in front of it. This generic WCF layer or Web service layer is easily accessible from Java oriented appplications and those Java oriented applications will talk on our Unix and Linux machines. The financial data residing in Oracle and SAP and all our human resource things also in SAP, can be easily coupled now with our WCF and Web service layer.
Now we have all the pieces to make even the scattered data in Outlook an integral part of our information management strategy. Every email gets saved into sharepoint under certain conditions and this information is instantly accessible from out my other Office applications. The same goes for Word, Excel, SAP and Oracle forms. With my WCF and Web service layers talking to the Sharepoint world, I put all the needed information from all my business applications into Sharepoint after creation. Everything in Sharepoint is accessible and searchable after about 5 minutes when it is inserted. For every application I have in every department in my organization this 5 minutes thing will be the case. No document, no email, no worksheet, no SAP form, no Oracle forms entity really nothing is there that is not connected to Sharepoint through the WCF and Web Service layer and everything talks with everything. Don’t forget that behind that WCF and Web Service layer is the server and client object model of Sharepoint.
Let is look at the new picture.
This is common sense but this common sense has become a bit more complex than it was in the past. So this common sense must be trained and feed with the proper knowledge. Look at the pieces in the picture, read the things mentioned in this blog and especially read the link about Change and Change management and this will be your roadmap to knowledge. If you as a CEO can say that every piece of information needed and mentioned in this blog is covered with knowledge, than you have your team ready. Read the paragraph where I mention basic things about Sharepoint, the knowledge about lists, content types, workflows and what not and buy that from outside or get your people on that level. They need it when importing and structuring all the information from out all your scattered information silos.
After that you need the proper knowledge to keep Sharepoint up and running. And you need the knowledge to make the WCF and Web Service layer. If you have great designers who can dream about the proper UML diagrams and who can do magic with C#, WCF, Team Foundation Server and Visual Studio in combination with the Sharepoint object model, then you are King. Don’t you have this in house? Buy it. Don’t you have it and you cannot buy it? Don’t start with this and put your energy in getting your organization at the right level.
Make this picture your goal and keep up that vision “ We want all our information accessible any time and any place and we want to do that with Sharepoint.”, which could be your strategic roadmap for the coming years. Start hiring brains now to get you there in the future. This blog was about Sharepoint and if you think about totally other products to do this, I think the essence stays the same. My experience is that Microsoft at this moment has one of the best decks of cards available with an impressive code base. This can work in our financial favor and it could mean organizational stability in the future.
Get your scattered data at one place and knock’em down.
Just be careful out there.
A. (2011). Creating Office Business Applications in Microsoft SharePoint 2010. Retrieved from Sharepoint Pro: