Section 10.4.  Open Source Empowerment

10.4. Open Source Empowerment Building an IT department with the power to envision solutions, evaluate options, and lead evolutionary implementation is the best investment a company can make if IT is a strategic factor. The more an IT department is supporting stable systems, the more buy-leaning it should be, and the more IT becomes a commodity. The more strategic value that IT brings, the more likely it is that flexible and dynamic systems must be supported, and the more build-leaning a department should be. The more build-leaning a company is, the more it must take responsibility for its own destiny and create and maintain skills for requirements gathering, architectural design, and development. Open source transforms companies, because it prepares them to be more build-leaning. As use of open source increases, IT departments take more responsibility as architects and creators of the solutions their companies need. (Doc Searles has an approach called "Do It Yourself IT" in which he recommends becoming more build-leaning.) To become more build-leaning, an IT department must develop a central architectural brain for crafting initial requirements, selecting from all the options available for implementation, and then leading the evolution of a better solution. Creating a well-oiled IT department is difficult, whether a department is buy-leaning or build-leaning. If an IT department is buy-leaning, it had better have a brilliant understanding of requirements so that it does not find itself locked into inflexible solutions that do not meet the company's needs. If an IT department is build-leaning, it had better have rock-solid education and documentation practices to protect the department from key-person risk. Most successful IT departments combine elements of these two paths. The best IT organizations are confident technology evaluators and component integrators. They have the skills to experiment and bring in partners to perform focused tasks. They do not ask anyone to solve their problems for them. They do not need one throat to choke. They have the creativity and drive of open source and the operational discipline and focus on process of traditional IT. The more skills that an organization develops, the more open source will be a viable part of the solution. As we pointed out earlier in the book, using open source exclusively is probably appropriate only in extreme cases. Vendors will always be part of the picture of a healthy IT infrastructure, but the IT department must call the shots. This book recommends that IT departments should become more build-leaning through open source adoption. First, in the right circumstances open source can lead to significant cost savings immediately. However, the following reasons apply as well: Acquiring and maintaining beginner- and intermediate-level skills allows for the use of open source of all maturities. Having open source skills in-house opens the door to using the most mature and productized open source to support stable systems, perhaps replacing commercial alternatives and saving money. For flexible and dynamic systems, using open source as the foundation for these applications and investing in skill building creates a stronger department rather than a richer vendor that is ever more deeply locked in. Highly skilled IT departments attract better-quality staff members, because talented technical people like being in a creative environment, surrounded by people from whom they can learn. Expanding the use of open source reduces vendor lock-in and increases available choices to solve problems. Now, let's examine the implications of becoming more build-leaning through open source adoption. 10.4.1. Creating a Learning Culture A build-leaning IT department is a learning organization in many ways and must be set up so that learning by the staff is a fundamental value and learning from experience is an ongoing process. Learning by the staff is vital to institutionalize skills and create a culture of creativity and empowerment. Keeping skills alive and spreading them around happens only when knowledge is shared and people put their skills to work. To institutionalize skills, an IT department should take most of the following steps: Encourage ongoing education Allow ideas to bubble up from the staff Encourage people to download and test software Encourage people to contribute to free software Maintain open communication channels with the community Encourage staff to think of themselves first as members of a professional community, not as members of the IT department The more that these characteristics are embraced, the more an IT department will become a dynamic force in its company. Why? Because the minds of the IT staff will be fully turned on. They will be engaged emotionally and intellectually in their work. Of course, this focus on learning has to be governed by the department's needs. IT departments are not pure research institutions. They are also not no-research institutions. It is up to the department's leadership to find the right balance. Learning as a process in a buy-leaning IT department is focused on getting a clear understanding of business requirements. Often, IT departments are far too confident that their requirements are correct, and they find out only after implementation that they were terribly wrong and that the system doesn't help its users. Even after an IT department realizes it won't get it right the first time, there might be no time or money left for an evolutionary process to create a complete solution. Another common problem is that the business organization frequently is not prepared to play the required role in evolution and, instead, expects a perfect solution on day one. Sometimes technologists in the company oversell what the solution can do and underestimate the work involved. Vendors consciously or subconsciously go along with all of this. Unfortunately, the past 40 years of software development have determined that predicting requirements is a losing proposition. The waterfall development methodology based on studying requirements beforehand and building a system to meet them has been thoroughly criticized. The message of Agile development methodologies such as the Rational Unified Process, eXtreme Programming, SCRUM, and others is that it is best to take a humble view of one's ability to predict requirements, and instead to develop the skills to build software and rapidly evolve it. Iterative development works, because nobody is asked to construct a long-term prediction of what requirements will be and then stick to it. Instead, a short-term prediction is made, and then modified based on experience using the software. In a build-leaning IT department focused on open source, the process of creating a solution for the business is always an iterative one. A department focused on learning from experience cuts the scope and spends much less money on the first version of the solution, knowing that the most important task is to get the application in users' hands so that learning can begin. Build-leaning departments allocate much more of the budget to resources for evolution than traditional IT. 10.4.2. Engineering Practices This model of doing more development for yourself falls apart completely if the open source being deployed is not matched with strong engineering practices so that future generations of engineers in the department can understand and maintain it. Business executives responsible for IT would no doubt be frightened by the prospect of technologists creating custom-built applications and integrations that might easily become unsupportable. When this happens, if the developer who understands it leaves, that's when the systems integrators and vendors have to ride to the rescue. What are strong engineering practices? All the things departments know they ought to do, but don't: Use source code management rigorously Build scripts that start by checking source code out from the authoritative repository, and then build the executable Deploy scripts to move versions in and out of production Maintain documentation at appropriate levels Rotate staff so that many people gain experience and skill with all of a department's key applications Conduct rigorous testing and quality assurance Conduct bug and issue tracking for technical support, and ongoing requirements engineering Properly separate development and operation control of production systems In facing these issues, an IT department becomes ready to support its role as a development organization. Consciously or subconsciously cutting corners on engineering practices is a surefire path to an open source quagmire. 10.4.3. Building a Better Staff Open source is not all peaches and cream. In the place of vendor lock-in, IT departments depend on maintaining a broader set of skills and more development capacity than most IT departments are used to. Pretending this is easy is a huge mistake. Finding the right people requires focused effort. One of the reasons that keeping technical skills in-house has such a bad reputation is that IT departments so frequently fail, and then systems integrators and vendors have to bail them out. Open source skills are only part of the successful build-leaning IT department, however. The goal is to build an IT staff that is capable of understanding business requirements from top to bottom, and also can understand the available solutions in great detail. The best IT department has a strong understanding of business management and of those jobs focused on gathering requirements. This business requirements braintrust works closely with the technologists who have a deep understanding of the software used at the company. The technologists have to be highly skilled and intelligent to understand the details of the products they are using. It is the rare technologist who obtains this understanding by reading. Technologists obtain it through experience, and for this to happen, they must be working on the software and doing development. Keeping the right skill levels requires a steady stream of work. This does not mean doing everything yourself; it means doing only some of it yourself. Maintaining such a department means providing a steady stream of interesting work for the technologists. While this model could work perfectly with commercial software, it is harder to achieve. Open source technology is simply more exciting to technical talent than commercial technology is. Once a talented technical team forms, for whatever reason, to work together on challenging business problems, open source will likely find its way in. What types of people will be attracted to an empowered department? Most frequently, it will be creative, talented people who want to put their skills to work and learn from their colleagues. Once the core of an IT department has formed, word gets around pretty quickly, and attracting a talented staff requires less effort and becomes more self-sustaining because talent is drawn to the department. 10.4.4. Increasing Choice, Reducing Vendor Lock-In Every IT department hates its vendors, or at least is frustrated by them. Most frequently this is because the software does not do exactly what the IT department wants it to do, and every time you want to change it you must pay some sort of consulting or licensing fee. Most of the time, the problem started during the sales process, when any mismatches between the IT department's requirements were papered over to get the deal done. IT executives who are excited about a product's functionality are all too willing to participate in this process. One of the big attractions of open source is that it helps avoid vendor lock-in, which is another name for being stuck with a product that does not meet your needs adequately. Once IT departments decide on a product, it must last for a long time. Empowered IT executives are in control of vendor relationships and are not beholden to them. Why? Because a build-leaning department has more choices. The vast potential of open source provides excellent leverage in negotiations with vendors. If you don't like what the vendor is offering, usually an open source alternative (or enough help from open source components that you can build what you want) is available. The more skill a build-leaning department has and the better able it is to learn, the less it needs to depend on vendors for consulting services. Open source can also be used for prototyping so that requirements can be better understood before a decision about lock-in must be made. Lock-in is not avoided completely, however. The replacement for vendor lock-in is a lock-in to the skills required to maintain the software being used. But investment in this lock-in means investment in a more powerful staff. Most vendors feel that open source is great for lowering the cost of providing their solutions, and that it can be an excellent companion to their products. However, they will invariably add, it is a completely crazy, unreliable, horrible choice as a replacement for their products. Open source is the vendor's friend when it promotes sales, but it's the vendor's enemy when it threatens sales.