There are many responses to such a question. The prominent amongst them are:
Denial: We have great researchers too. Look at Manindra Agarwal. China publishes more, but most of it is of poor quality. We did not believe in Publish or Perish and hence we haven't got into the habit of writing everything we do. The numbers does not tell the whole story. Our number of researchers is small, and so on.
Resources: We are a poor, third world country. What do you expect. The percentage of GDP spent on education is small, and on top of that our per capita income is small. The output per dollar is very high in India. (And this is always per dollar in every discussion, why not output per million Rupees. To me, this indicates that research invariably means solving American problems.) If we are given as much resources as a typical US university gets, we will solve all problems of
Industry: Our industry is low technology. They just want to do simple things. They are interested in buying technology when needed rather than solve their problems through research and innovate. Look at US, how industry works in partnership with universities to solve problems. Of course, people will then quickly add that industry is changing lately, investing in research, interacting with universities, but to get to the US level, it will take some time. (And once again, why US level is the benchmark. I think the top institutes in India are already doing better than US.)
Quality and Quantity of faculty: We have such a shortage of quality faculty. As a result, we are stuck with not so great faculty, and then we ask them to handle 2 courses a semester. How can they do high amount of teaching, take care of academic administration, and also do cutting edge research, when their own scholarship is suspect.
Any description of the problem of lack of research excellence in India will be like a description of an elephant by a bunch of blind men. All of them have some validity but the bigger picture is something else.
And still, as yet another blind person, I will put in another perspective.
The structure of research funding in India is such that research is a huge loss making exercise for the universities that carry out research. If you submit a proposal to a funding agency, you can only charge 16.7 percent overhead for managing the project. And since it is expected that the management costs don't rise linearly with the size of the project, some funding agencies will cap the overhead portion of the budget. In the budget that I prepare, I can not put the salary of permanent employees (like myself), rent for the space used by the project, electricity and other such resources used by the project, and so on. The logic is simple. Another arm of the government (MHRD, AICTE, UGC, or state government) is typically paying for the salary and infrastructure. So why not keep getting money from them and not from funding agency. The problem in this logic is that it is essentially saying that only government institutes must do research. Private colleges should not dream of doing research. Now, if this was the national policy which every one understood loud and clear, it would still be a problem but at least we would understand the source of the problem. But the problem is that other government agencies who are in the business of evaluating the quality of higher education do not believe in this national policy. Your accreditation by NAAC or NBA would crucially depend on your research output. UGC will keep reminding you that every promotion of faculty members should be based on research output.
Can we have DST/DBT/DEITY and other departments funding research sit jointly with UGC/AICTE/MHRD/NAAC/NBA and others involved in maintaining and judging quality of education and decide for once and for all whether the research is supposed to be done only by government universities or also by private universities. (And let us not forget, a vast majority of higher education is in private sector today.) If private sector should do research, can we have an overhead that can pay for the part salary of faculty and for the infrastructure needed to carry out the research. (It would mean doubling or tripling of the overhead. For a 100 rupee project, the overhead should also be close to 100 rupees.) On the other hand, if they agree that the vast portion of higher education system should not be expected to do any research, NAAC and NBA types of bodies should stop asking questions on research output.
If I have to do research, someone got to pay my salary. If research funds can not be used for the same, then the project is a loss to the college. Someone got to invest in the buildings and other infrastructure. All that is loss to the institute. So what is normally done. To an extent, we will inflate the budget - if I need one server, I will write two or even three, just to give an example. So the tuition will pay my salary, and some of it will be recovered through fraudulent invoicing, which every project monitoring committee will be aware of. But of course, you can commit only so much of a fraud. The rest of the research money comes from tuition. Now, tuition in India is already very low compared to the cost of providing quality education. So we anyway have very poor quality of education in most of our colleges and universities. If you declare that the cost of research should also be borne from the tuition, well, the consequences are for all to see.
Paying only a portion of the costs for research projects also mean that there is typically very little accountability for the research output. If the funding agency gave 50 lakh rupees, and the cost of salaries, and infrastructure is also 50 lakhs, then the funding agency can not dictate, at least not morally, that the output of the project should be commensurate with an investment of 1 crore rupees.
The second problem is Measurement of Research. The best way to measure research is through peers. However, our regulatory bodies are only interested in numbers. This has created such a racket in the country of fake journals, and fraudulent conferences. If we will keep insisting on numbers, people involved will be happy to game the system to get all the goodies (promotions) that the system will give to those who are performing well as per that metric. Of course, this problem is not just restricted to poor quality colleges. Even at the top level, there are cliques. If my friend applies for a grant, he gets it even if the quality of proposal was poor and the CV of the friend (the proposed Principal Investigator) was poor. And in all this, people who are expert in one area will gladly decide the quality in areas where they have very little knowledge.
The other major problem is Cultural. If we have a problem, we want to solve it quickly through a low-cost innovation which will solve our immediate concern, but may generate other problems. Studying all aspects of a problem, and figuring out all solutions, comparing them, considering their side effects, and so on is too boring for us. Also, we are not respectful of intellectual property. School teachers will tell the students that it is ok to copy from wikipedia or any other Internet resource. If we don't value intellectual property, it would be very difficult for us to be motivated to create intellectual property. Research also means looking at multiple options and having a belief that any of those options could be the best. This mindset requires one to be open to new ideas, new theories, and the possibility that what we have believed all along may turn out to wrong. Do we have such a mindset today. Do we accept criticism of our beliefs easily. Another cultural factor that I heard today in the discussion was being risk averse. Poor people are generally risk averse and 99 percent of people in India would consider themselves in a bracket where they want some stability in income, and would not take risks. If this is the background of most people, is it rational to expect that people who get into either a researcher's career or are into research management would take risks.
So, if we want to be a research powerhouse, we will have to do a serious introspection about our cultural upbringing, which means a major responsibility on all educational institutes, including K-12 schools, and not just institutes of higher learning. And we will have to bring in the management practices that ensure that research is not a hugely loss making exercise.