Confusing debug=timing parameter

[S G]

Hi,

I am using Solr 4.10 and its response time for the clients is not very good.
Even though the Solr's plugin/stats shows less than 200 milliseconds,
clients report several seconds in response time.

So I tried using debug-timing parameter from the Solr UI and this is what I
got.
Note how the QTime is 2978 while the time in debug-timing is 19320.

What does this mean?
How can Solr return a result in 3 seconds when time taken between two
points in the same path is 20 seconds ?

{
  "responseHeader": {
    "status": 0,
    "QTime": 2978,
    "params": {
      "q": "*:*",
      "debug": "timing",
      "indent": "true",
      "wt": "json",
      "_": "1481992653008"
    }
  },
  "response": {
    "numFound": 1565135270,
    "start": 0,
    "maxScore": 1,
    "docs": [
      ....
    ]
  },
  "debug": {
    "timing": {
      "time": 19320,
      "prepare": {
        "time": 4,
        "query": {
          "time": 3
        },
        "facet": {
          "time": 0
        },
        "mlt": {
          "time": 0
        },
        "highlight": {
          "time": 0
        },
        "stats": {
          "time": 0
        },
        "expand": {
          "time": 0
        },
        "debug": {
          "time": 0
        }
      },
      "process": {
        "time": 19315,
        "query": {
          "time": 19309
        },
        "facet": {
          "time": 0
        },
        "mlt": {
          "time": 1
        },
        "highlight": {
          "time": 0
        },
        "stats": {
          "time": 0
        },
        "expand": {
          "time": 0
        },
        "debug": {
          "time": 5
        }
      }
    }
  }
}