For the HiPS client multiple tiles have to be fetched for time efficiency. To achieve this, we create a separate thread for each outgoing request. Thus, requests are sent concurrently. A comparison is done utilizing Python’s threading
library. The elapsed time is calculated using the time
module. For fetching the tiles urllib
, grequests
, aiohttp
, and asyncio
packages are used. The HiPS survey chosen for this comparison is alasky.u-strasbg.fr
.
For fetching 10 tiles, it takes the following time (in seconds):
Elapsed Time URLLib (without concurrency): 3.5430831909179688
Elapsed Time URLLib (with concurrency): 0.388397216796875
Elapsed Time URLLib (with aiohttp): 0.3900480270385742
Elapsed Time GRequests: 1.6238431930541992
Similarly, for fetching 100 tiles, it takes:
Elapsed Time URLLib (without concurrency): 37.7027428150177
Elapsed Time URLLib (with concurrency): 5.575664043426514
Elapsed Time URLLib (with aiohttp): 2.4697625637054443
Elapsed Time GRequests: 4.273705244064331
The pros of grequests
is that it takes less time when large number of requests have to be sent. But urllib
(with threading) gives a better response time when requests are few.
Using aiohttp
with asyncio
seems to be the best option. Its response time is almost 50% less than grequests
.
The source code for this comparison can be found here.