Marketplace Experimentation and Democratizing Innovation

Last week's discussion focused on execution in the marketplace, particularly in the context of e-business at IBM. The core strategy involved leveraging existing assets—talent, clients, and products—with new internet-based technologies. This approach framed the internet, World Wide Web, and e-business as tools to enhance the value of existing offerings, rather than replacing them entirely.

Key aspects of organizing this market strategy included balancing proprietary and open systems, in-house development versus partnerships, defining key offerings, and establishing metrics for progress. A crucial point was the need for focused effort when dealing with disruptive technologies like the internet, which impacts various layers of a business. The emphasis was on taking incremental steps, starting with "first base" rather than attempting a complete transformation all at once.

Customer-Based Experimentation

The discussion then shifted to customer-based experimentation, highlighting the work of Tom Key and Eric von Hippel. Experimentation in the marketplace is vital for innovation, especially when moving technologies from the lab to real-world application.

The Changing Landscape of Experimentation

Historically, marketplace experimentation was prohibitively expensive due to the high cost of building prototypes and the reliance on human-centric channels for client interactions. This often meant launching a product and hoping for market acceptance. However, this is changing due to several factors:

• The Internet: The internet serves as the world's largest platform for innovation, experimentation, and collaboration.
• Inexpensive IT Systems: The cost of IT infrastructure has significantly decreased. New ventures require far less capital than during the dot-com era, largely because computers and open-source software (like LEMP and LAMP stacks) are now very affordable. Even supercomputing access is more attainable.
• New Tools: A wide array of new tools enables more efficient experimentation.

Tom Key emphasizes that companies must integrate experimentation into their fundamental structure from the outset, rather than treating it as an afterthought. This means organizing efforts around extensive experimentation, such as using small "black belt" teams for rapid prototyping and early testing (alpha or testbed releases) to quickly refine ideas.

The pace of innovation has accelerated dramatically. Product plans that once spanned three to five years are now considered impractical for software or services, as markets change too rapidly. The recommendation is to focus on what can be achieved in six months and iterate quickly. While not universally applicable (e.g., semiconductor fabs still require long-term planning), this iterative approach is crucial for many modern products.

Experimentation Across Industries

The concept of rapid experimentation and "failing fast" is gaining traction beyond the IT and software industries.

• Pharmaceuticals: While drug development still involves long timelines (e.g., 10-year product plans), there's a growing emphasis on accelerating testing of new pharmaceuticals using advanced technologies. The challenge lies in the high cost of each iteration.
• Retail: A significant percentage (70%) of new retail products fail because customers don't buy them. The earlier a failure is identified, the less costly it is. Quick failures are learning experiences; prolonged failures after substantial investment are significant problems. The goal is to quickly identify what doesn't work to focus resources on promising efforts.
• Other Technologies: New drug sequencing technologies, genomics sequencing, and advanced simulations are enabling faster testing and design validation across various fields.

The overarching theme is to reframe business processes to embrace this iterative, experimental approach, starting with small opportunities and refining them incrementally.

Leveraging Existing Components and Partnerships

To accelerate market entry, businesses should:

• Leverage Existing Infrastructure: Utilize the internet and open-source platforms as much as possible. Open source offers a cost-effective starting point, even if scaling later might require proprietary solutions.
• Build on Existing Components: Avoid starting from scratch. By using well-understood existing components, the overall probability of success increases, and risks are minimized. Improvements to these components can be made in later releases.
• Strategic Partnerships: Partnering with entities that have an established base or platform allows for faster market entry. This contrasts with the older IBM mindset of building everything in-house, which often led to lengthy development cycles and missed market opportunities.

This attitude of leveraging existing bases is a key reason why initial investments are much lower today.

Low Fidelity Designs and Unique Innovation

Tom Key's third point emphasizes "low fidelity" designs, meaning early designs don't need to be perfect or final. They can be coarse designs that are incrementally improved based on market feedback. This is another way of advocating for iterative design.

Finally, businesses should clearly identify their unique innovation and focus energy and risk-taking on that specific new element. By building new innovations on top of well-understood, low-risk existing components, the overall probability of success is enhanced. This approach helps manage complexity by reducing the number of "moving parts" in a project, allowing for incremental growth and later replacement of core components.

The "80% Solution" Dilemma

A critical consideration with early releases is the "80% solution." While getting a product out quickly is beneficial, releasing something perceived as incomplete or flawed can damage credibility. The challenge lies in finding the right balance—the "line" between sufficient functionality and unacceptable incompleteness. This requires strong intuition from design managers and chief architects, intertwined with market understanding.

It's also important to distinguish between internal failure and public release. Simulations and internal testing can identify many flaws before a product reaches the public. For instance, concept cars are used to gauge public reaction before significant investment in production.

Market Experimentation in Software

In software development, rapid prototyping and mock-ups are common, allowing for quick user feedback. Alpha and beta programs are crucial for this. IBM's "alphaWorks" initiative, for example, provided a channel for research lab ideas to be tested in the market by making them downloadable, with legal agreements to prevent commercial use during the experimental phase.

Companies like Google extensively use "beta" labels for their products (e.g., Gmail, Google Maps), which serves as a marketing strategy to manage user expectations and allow for continuous iteration. The ability to quickly roll back changes is key to this rapid experimentation model. This approach requires an architecture designed for rapid recovery, rather than monolithic systems.

Democratizing Innovation (Eric von Hippel)

Eric von Hippel's work on "Democratizing Innovation" highlights a shift from manufacturer-based innovation to user-centric innovation.

Manufacturer vs. User-Based Innovation

• Manufacturing Innovation: The traditional model where a manufacturer gathers information, develops a product internally, and then pushes it to the market. Users are passive consumers.
• User-Based Innovation: Users actively participate in the innovation process, sometimes with manufacturer support, sometimes independently (e.g., hacking, modifying products). This model is increasingly important, especially in the early stages of product development where the exact nature of the product is still unclear. Leading-edge users are crucial partners in this phase. As products mature and scale, traditional manufacturing principles become more dominant.

Examples of User-Based Innovation

• Supercomputing at IBM (early 1990s): IBM, developing parallel supercomputing architectures, partnered with leading research institutions like Argonne National Laboratory and Cornell Supercomputing Center. These institutions, accustomed to influencing design, became integral to the development process. They received early, incomplete versions of the machines and provided critical feedback, effectively acting as an extension of IBM's development team. This collaboration allowed IBM to refine its product (eventually the SP2) and gain market experience before a full commercial launch. This demonstrated the value of engaging lead users who are pushing technological boundaries.
• Gaming and Virtual Worlds: The gaming community, particularly massively multiplayer online games (e.g., World of Warcraft) and role-playing games (e.g., Second Life), are leading indicators of future technological trends. IBM recognized this and became active in virtual worlds based on studies from its Academy of Technology.
• Social Networks: Platforms like Facebook, MySpace, and Craigslist originated from users creating solutions for themselves or their communities. These user-driven innovations eventually attracted commercial interest, demonstrating the importance of observing and engaging with user-generated solutions.

The Knowledge Gap: Users vs. Manufacturers

Eric von Hippel emphasizes the differing knowledge bases:

• Users: Are experts on their requirements. In nascent markets, information about needs resides primarily with leading-edge users, not in traditional market analysis data.
• Manufacturers: Have expertise in what they can build. The challenge is to bridge the gap between what users need and what manufacturers can provide. Collaboration with lead users helps align these two perspectives.

The Blurring Lines Between User and Manufacturer

In many contemporary examples, especially in software, the distinction between user and manufacturer blurs. Mark Zuckerberg, the creator of Facebook, was a user himself. Craig Newmark, founder of Craigslist, also created the platform for his own needs. In the supercomputing example, IBM opened up its infrastructure, allowing users to become "manufacturers" to some degree by contributing to the development.

This convergence is particularly evident in software and open-source projects, where users actively contribute to development. While not all users can "manufacture" complex physical products (e.g., fuel-efficient cars), they can still provide valuable feedback and even modify existing products for new uses (e.g., Coast Guard modifying Cisco routers). The key distinction lies in the benefit: user innovation is driven by solving personal problems, while manufacturing innovation is driven by selling solutions.

Why Now? The Shift to a Knowledge Economy

This rise in user innovation is attributed to several factors:

• Shift from Industrial to Knowledge Economy: In an industrial economy focused on physical goods, user involvement is harder. In a knowledge-based economy, where much innovation is captured in software, user collaboration is much easier.
• Enabling Technologies: The internet, collaborative technologies, and the ability to distribute software facilitate distributed collaboration that was previously impossible.
• Innovation in Services: Historically, engineering focused on automating back-office processes or physical tasks. However, new technologies now enable innovation at the "services layer" of businesses and economies. Services involve people interacting with each other (e.g., doctor-patient, employee-employee). Instead of automating these interactions, technology can provide tools to enhance them (e.g., genomic data for doctors). This allows for user innovation in areas where people are directly involved in providing or receiving services.

This profound shift means that the lines between users and manufacturers are becoming increasingly blurred, fostering a new era of collaborative innovation.

Communicating with the Market

Effective communication with financial analysts, IT analysts, and the press is crucial. While marketing professionals play a role, technical experts are often best equipped to explain complex, disruptive innovations due to their in-depth understanding. They can articulate messages effectively and respond to questions with depth, which analysts and the press can discern. Failing to engage effectively can lead to competitors gaining recognition for similar or even inferior products.