Anaemic economic growth weakened the demand links and incentives that spur innovation; and after the initial flush of idealism that created a host of dynamic institutions, the heavy hand of the Indian state rendered many of them plodding bureaucratic sloths.

For a country with one of the largest concentrations of scientists and engineers, India does poorly on conventional indicators for measuring the level of innovation in a country. China has clearly outstripped India in patent filings, but even Australia, with just 2 per cent of India’s population, files more patent applications in the United States than India.

India’s performance is somewhat better when it comes to publications in science and engineering in international refereed journals. The pace of research activity in India has picked up, with emerging leadership in several research areas, especially chemistry and, to a lesser extent, engineering biology and biotechnology.

Of course quantity is not quality. Most patents see little activity and only a small fraction of journal articles make a significant contribution. While in quantitative terms India’s performance pales behind the scorching pace set by China, India does considerably better than China in quality terms (as measured by citations per article). The rate of improvement has been limited by a decline in the number of Indians doing doctoral work in science and engineering in the most demanding market — the United States.

A lack of innovative dynamism has also been noted with regard to the absence of genuinely new global products introduced by Indian companies. Few Indian companies are perceived to have the capacity to innovate disruptive technologies.

In contrast to India’s lacklustre performance in conventional measures, other sources are almost breathless about the seemingly boundless creative energy in India. Headlines like ‘India the world’s secret silicon valley’ and ‘India’s innovation stimulus’ herald a new type of innovative activity popularised variously as ‘frugal innovation’ or ‘Gandhian innovation’.

These statements point to a certain dynamism and creativity that is distinct from conventional measures such as publications and patents. What is the nature of this creativity and innovation and what are its sources? Why is India doing well on some (less conventional) indicators of creativity but much less well on more conventional measures?

One hypothesis that reconciles this discrepancy is that the nature of innovation has changed. As companies move away from having one large research and development (R&D) centre in their home country to a more distributed global model, India is emerging as a global innovation hub. Multinational corporations have established more than 600 captive R&D centres across India. These include not only centres for information technology firms such as Google and Microsoft which are drawn to India’s specialised knowledge, but also engineering firms such as General Electric and Philips, and increasingly pharmaceutical firms. The reason why India is not ‘visible’ as the source of this R&D is that these India-based R&D centres are part of an intra-firm R&D supply chain that is visible only to other business units of the firm.

India’s emergence as an important R&D hub for many leading global firms stems from the increased availability of high-quality talent, both within India as well as the greater willingness of overseas Indians to move back to India if the re-entry mechanism is a cutting-edge R&D centre of a leading company.

While much of India’s R&D is for products serving the global market, there is also a burgeoning branch of innovation focused on modifying global research to Indian conditions. Indian agriculture is a good example.

With the productivity gains of the Green Revolution technologies waning and increasing stress on soils and water, Indian farmers are desperate for new technologies. But the travails of India’s agriculture universities and the virtual collapse of the public sector extension system mean that, unlike in the 1960s and 1970s, farmers have to look to the private sector. The most significant productivity improvements in the performance of the agricultural sector have come from yield increases brought about by new production technologies, with Bt transgenics in cotton and single-cross hybrids in maize the most visible examples. These advances occurred despite weak support from the Indian state and fierce opposition from Indian non-government organisations (mostly urban).

However, conventional state-led entities like the Indian Council of Agricultural Research and agriculture universities are playing a much weaker role. Public research and extensions systems are increasingly unable to address farmers’ needs. Instead, creative responses come from the private sector. Both multinational corporations and a host of local entrepreneurs respond to farmers seeking solutions. One example is using cell phones to offer illiterate farmers farm advisory services.

The poor quality of public services is leading to a range of delivery-model innovations. In drinking water, for instance, in place of a traditional water utility with complex capital-intensive infrastructure such as pipelines, metering, and billing systems, attempts are being made to meet the needs of people in India’s villages using ‘micro-utilities’, often operating as franchise models. These distribute water over short distances and above ground where quality is easily monitored and drinking water separated from water for other household uses.

There are several forces shaping creativity and innovation in contemporary India. First, the sheer heterogeneity and diversity of Indian society makes it a fertile ground for ideas and creativity.

A second powerful force is demographic: the combination of a huge population, half of which is under 25; the entry of newly empowered socially marginalised communities into the mainstream of Indian economic, social and political life; and dramatic changes in the aspirations of India’s young, empowered by new media technologies.

The third driver is rapid changes in technology. This is allowing even those Indians with modest resources to leapfrog technologies by leveraging free open-source software such as Skype, cloud computing and cloud-based office tools. The ubiquity of cell-phone ownership means that it has emerged as one of most egalitarian technologies, sparking creative, low-cost solutions for a range of challenges India faces.

Last but not least is the Indian state. Regrettably, its poor performance puts greater pressure for ideas and innovations on issues that it should be addressing but is unwilling or unable to do. Where the state retains its monopolies — in defence, security, land allocation, urban utilities and mining — there is considerably less innovation unless new entities are created outside the traditional government apparatus (exemplified by the recent success of the Delhi Metro and possibly the Unique ID project).

Sadly, one potential source of new ideas and innovation — universities — is severely underperforming. Innovations come from small purposeful research centres, narrowly focused institutions and think tanks, but rarely from the hundreds of Indian universities.

Will the ideas, creativity and innovation emanating from the India’s robust private sector and civil society translate into organisations capable of transformative actions of sufficient scale to compensate for the manifold weaknesses of the Indian state? Or will the absence of complementary capabilities and actions by the Indian state drag down the private sector and the burgeoning numbers of social entrepreneurs as well? That might well be the central question around which India’s future will hinge.

Devesh Kapur is Professor of Political Science and Director of the Center for the Advanced Study of India at the University of Pennsylvania.

This article appeared in the most recent edition of the East Asia Forum Quarterly, ‘Ideas from India’.