I had an email exchange recently with an individual who is an advisor in our work. It started with my sending him an email referring to a person I had come across - a polymath who exhibited interest and hands-on work in areas far from his chosen field of expertise (without identifying the individual, imagine a doctor who potters around with classic sports cars and knows how to put them together from scratch, not kits, with great finesse - I did know one such individual during my days in the US). In my own exuberant manner, I exclaimed "this is the kind of person we need!". To which he responded, trifle drily, "he is simply a hobbyist. There is a gulf of difference between a hobby toy and a real machine". So I reflected on what the gulf of difference could be and if multi-dimensional interest and experimentation was of no consequence in today's world.
Here is how I see this. Most of the companies in the world pursue opportunities related to products or services delivered in some unspectacular way. I call this block and tackle execution (BATE), meaning they are only doing what is a necessary business and what really differentiates one company from another is how well they do it. A few are great, many average, and most are mediocre. Wikipedia defines block and tackle as "characterized by the use of a single continuous rope to transmit a tension force around one or more pulleys to lift or move a load".
Larsen & Toubro, for instance, does a lot of engineering involving building structures for nuclear reactors, submarines, oil refineries, and sundry infrastructure work. These projects are often complex, require great expertise and demand thorough domain knowledge from its engineers. They are regular engineering nonetheless - civil, mechanical, or electrical for the most part, with practices that are well understood and going back several decades. Nothing particularly novel or entirely new in scientific discovery is demanded here. Such companies pursuing everyday execution use their vast resources including  specific expertise and long experience ("single continuous rope") to leverage work in many different, but allied, areas  of endeavor ("transmit a tension force around one or more pulleys to lift or move a load"). In other words, BATE companies enjoy what in economics is referred to as economies of scale resulting from initial investment cost and operating labor cost and economies of scope resulting from opportunity to exploit knowledge across project boundaries within the company. L&T is not alone in this; Schlumberger, Bechtel, Toyota, Ford, Tata Motors, Mitsubishi, Hyundai, American Airlines, Pepsico, to name a few - in fact, most organizations, come to mind. They are…all BATE companies whose daily grind comprises having the work done at high efficiency, getting the product out at low cost, and ensure delivery using established and time-tested knowledge, processes, and systems.
And then there are companies such as Corning, Intel, Qualcomm, Google, Amgen, Genentech. What's the difference? At a very obvious level, these "technology" companies spend between 10% and 25% of gross revenues on R&D. Mainstream, or BATE, companies spend less than 10%, generally under 5% if at all. This is not to say that Toyota or Ford do not have extensive research and development. They do, only it is not fundamental to their business the way it is for Intel, for example, where one sinks or swims with the duration of a product's lifecycle and the speed expected to come up with new products (ie, shortening product lifecycles). We could refer to such companies as "Innovation-Driven Product Development" (IDPD) companies.
Just so we are clear, automobile companies have increasingly short product lifecycles, but these are generally restricted to design, not whole product makeovers (engine, drivetrain, battery, etc) which still happens only every six or seven years. Contrast that with a semiconductor company facing makeovers every 18 months.
Another point to remember is that most BATE companies are often integrators - the discrete elements come from a variety of focused companies big and small, and some of them, depending on the criticality of the component or product they supply and where they are situated in the technological matrix, are possibly IDPD companies. An aerospace company integrates structures, avionics, jet engines, power systems, wheel assemblies, etc from a variety of high technology vendors such as Rolls Royce, Pratt & Whitney, Rockwell, Honeywell, Goodyear, etc. While the economics of traditional BATE industry is well understood, that of innovation is still in its infancy since the days of Schumpeter and uses much the same theoretical frameworks to understand it.
From a practitioner's standpoint, however, there is less mystery: it is based on a foundation of inspiration (observation and discovery), experimentation, analysis, and solution. The uniqueness and value of the solution determines the success of an innovation in product or service. Traditional BATE companies confine themselves to analysis and solution as they are based on well understood, extensively used,  repeatable or reusable technologies and processes. The high cost of R&D, then, is simply because of the entire and complex discovery path to innovation that IDPD companies have to traverse that is sought to be protected as intellectual property. Yet another way to understand this maturity vs immaturity of two entirely variant businesses is to look at standards: BATE industry has established standards while IDPD industry often experiences evolving or absence of standards.
Why is this important in the context of my email exchange? Verily because creativity in the lab does not equate that easily to long experience or even individual expertise. It requires knowledge, of course, but equally an attitude of experimentation, multitasking, cross-domain enquiry, risk tolerance, and discovery. The story of Twitter comes to mind. The venture came out of a struggling podcasting company called Odeo where Jack Dorsey, an undergraduate at the time, toiled away to build what became the 140-character technology that Odeo never envisioned. 3M's Post-it came off a similar serendipitous outcome when Art Fry made use of the company's officially sanctioned "permitted bootlegging" policy to develop an idea originally discovered accidentally by Dr. Spencer Silver who was attempting to develop a very strong adhesive. Google's much-vaunted search engine was a solution in search of a problem until it happened on a creative strategy many years after it was founded that allowed it to make humongous revenues. Remember, there were at least six other ventures who already had search products in the market when Google came onto the scene.
The point is that knowledge is a necessary, but not sufficient, condition for innovation. It further requires the spirit of  discovery and perseverance and a yen for multi-disciplinary experimentation with no visible payoff. These qualities do not necessarily exist within BATE organizations where engineers can bore down into a problem and solve by fitting their experience from another comparable problem they have come across before. My advisor friend is quite right in his implicit belief that expertise and experience deliver on execution. But off the mark when he says a hobby toy cannot make it to a real machine. That's judgmental and is the primary reason why companies cannot see their way through innovation. Hobbyists are unfit in block and tackle execution with its requirement of expertise and experience. But they are very much a necessity for the eclectic mindset required of creativity in IDPD organizations.
Remember Xerox PARC in Palo Alto? Xerox Corp HO 3,000 miles away to the east never quite realized that the PARC was the jewel in its crown and viewing it, in the words of an award-winning writer, with uneasiness and hostility. The PARC folks were not part-time hobbyists, of course, but those scientists had all the same characteristics of passion addicted to a hobby who challenged odds, refuted reason, and were willing to rewrite the trajectory of technology. They did so with out-of-the-box thinking, drawing insights from multiple areas, and taking an experimental, prototyping, approach.
Good BATE companies can develop and ship good products. Even great products. Only don't expect something tangibly innovative to come out that charts a different growth path for the company. For that, you need a multi-dimensional hobbyist team that is insane enough to persist and keep plugging away at something that no one sees any value in immediately. Top this off with a disciplined process and you have the basis for an innovation engine inside an IDPD company.
Note: I do not wish to give the impression that all BATE companies or all IDPD companies are engineering / tech companies. Far from it. Some of the best BATEs are organizations such as WalMart, FedEx, Starbucks, Wegman's, Sysco, etc - all inherently non-technology, though with a very strong technology underpinning. In the same vein, innovations can, and do, occur in areas such as business models, financial, distribution, etc.