Notes on software, astronomy, machine learning, and infrastructure.

Google Summer of Code 2018 final evaluation report

Link to GitHub repository: https://github.com/BoostGSoC18/geometry

The work is present under the following branches:

Summary

The goal of this project was to implement the direct and inverse geodesic algorithms in the Boost Geometry library. These methods were proposed by Charles Karney in his paper in 2011.

In a previous blog post, the inaccuracy of the existing methods was discussed, which provided inconsistent results for nearly antipodal points. To monitor the progress, a weekly report was provided through GitHub, which summarized the work done. Finally, benchmarks were performed against existing methods in Boost Geometry. The performance metric used was execution time and accuracy.

Additional material, such as utility scripts for parsing the …

Read more →

Using variadic templates with lambda expressions in C++ for constrained optimization

Constrained optimization problems are encountered in numerous domains, such as protein folding, Magnetic Resonance Image reconstruction, and radiation therapy. In this problem, we are given with an objective function which is to be minimized or maximized with respect to constraints on some variables. The constraints can either be soft constraints or hard constraints, which can be specified by boolean operators, such as equality, relational, and conditional operators.

This post provides insight on how to model constraints using lambda expressions, and how to pass a varying number of constraints to a function using variadic templates. Before moving on with the C++ implementation, it will be helpful to review how variadic functions are used in C and how they differ from the …

Read more →

Inaccuracy in Boost Geometry geodesic algorithms for nearly antipodal points

Nearly antipodal points or antipodes refer to the most geographically distant points on a sphere, that is, the points are diametrically opposite to each other. If a line is drawn between these two points, it passes through the center of the sphere and forms its diameter.

Computing the great circle distance between these two points is often a corner case for most geodesic computations, and the distance is either overestimated or underestimated. In case of Vincenty’s formulae, the solution fails to converge, or provides inaccurate results. This can have major implications in applications which rely on accurate results, such as flight navigation systems. The software can handle this either by doing an error analysis check and providing specific values …

Read more →

An overview of activation functions used in neural networks

An activation function is used to introduce non-linearity in an artificial neural network. It allows us to model a class label or score that varies non-linearly with independent variables. Non-linearity means that the output cannot be replicated from a linear combination of inputs; this allows the model to learn complex mappings from the available data, and thus the network becomes a universal approximator. On the other hand, a model which uses a linear function (i.e. no activation function) is unable to make sense of complicated data, such as, speech, videos, etc. and is effective only up to a single layer.

To allow backpropagation through the network, the selected activation function should be differentiable. This property is required to compute …

Read more →

Parallel tile fetching and CPU-and-memory statistics

The hips package now supports parallel tile fetching. The user can achieve this either using the urllib or aiohttp package.

In case of aiohttp, the fetched tile data is coupled with HipsTileMeta to create a HipsTile object. This ensures there is no misalignment of tile data, otherwise, tiles could get swapped during the drawing period.

async def fetch_tile_aiohttp(url: str, meta: HipsTileMeta, session, timeout: float) -> HipsTile:
    """Fetch a HiPS tile asynchronously using aiohttp."""
    async with session.get(url, timeout=timeout) as response:
        raw_data = await response.read()
        return HipsTile(meta, raw_data)

We also limit the amount of simultaneously open connections using aiohttp.TCPConnector class. The returned object is passed to aiohttp.ClientSession‘s __init__ method. This procedure can be understood in …

Read more →

Google Summer of Code 2017 final evaluation report

Link to GitHub repository: http://github.com/hipspy/hips

In addition to the main hips repository, I also maintained my personal HIPS-to-Py repository on GitHub. This contains Jupyter notebooks which showcase the functionality in hips and numerous related Python scripts. The Wiki page contains a short description on hips. It also contains links to resource documents and telcon notes, which are hosted on Google Docs.

List of Pull Requests

Work related with HiPS tile drawing
Read more →

Fixing tile distortion issue in hips package

As documented in the tile distortion issue section, the previous technique for drawing HiPS tiles brings some astrometry offsets for distorted tiles.

An example of such distortions can be viewed at this link (uncheck “Activate deformations reduction algorithm” to view the astrometry offsets): http://cds.unistra.fr/~boch/AL/test-reduce-deformations2.html

To overcome this issue, the parent tile is divided into four children tiles if it meets the following two criteria:

  • One edge is greater than 300 pixels when projected
  • Or, the ratio of smaller diagonal on larger diagonal is smaller than 0.7 and one of the diagonal is greater than 150 pixels when projected

For handling these checks, a function is_tile_distorted is introduced:

def is_tile_distorted(corners: tuple) -> bool …
Read more →

RGB tile drawing in hips package

The hips package now supports RGB tile drawing. To make this possible, the output image dimensions had to be altered according to the following configuration:

The output image shape is two dimensional for grayscale, and three dimensional for color images:

  • shape = (height, width) for FITS images with one grayscale channel
  • shape = (height, width, 3) for JPG images with three RGB channels
  • shape = (height, width, 4) for PNG images with four RGBA channels

In addition to this, in-case of JPG and PNG format, the tiles are flipped in the vertical direction, which leads to incorrect tile drawing using the previous technique. The figure below is taken from the hips paper, figure 6, which shows the inverted tiles.

HiPS inverted tiles figure

To overcome this, the …

Read more →

Parameterized testing using Pytest

Pytest provides a feature for parameterized testing in Python. The built-in pytest.mark.parametrize decorator enables parametrization of arguments for a test function. This allows the user to compare the values for input and output.

Here is a typical example which shows its usage:

get_hips_order_for_resolution_pars = [
    dict(tile_width=512, resolution=0.01232, resolution_res=0.06395791924665553, order=4),
    dict(tile_width=256, resolution=0.0016022, resolution_res=0.003997369952915971, order=8),
    dict(tile_width=128, resolution=0.00009032, resolution_res=0.00012491781102862408, order=13),
]

@pytest.mark.parametrize('pars', get_hips_order_for_resolution_pars)
def test_get_hips_order_for_resolution(pars):
    hips_order = _get_hips_order_for_resolution(pars['tile_width'], pars['resolution'])
    assert hips_order == pars['order']
    hips_resolution = hp.nside2resol(hp.order2nside(hips_order))
    assert_allclose(hips_resolution, pars['resolution_res'])

Without the support of parameterized testing, the code had to be duplicated three times …

Read more →

Creating custom decorators in Python 3.6

In the hips package, often data has to be fetched from remote servers, especially HiPS tiles. One way to cut back on the queries was by introducing the hips-extra repository. This contains HiPS tiles from various HiPS surveys. This allows us to quickly fetch tiles from local storage, which makes the testing process less time-consuming.

As hips-extra repository does not come with the standard hips package, user has to manually clone it. The availability of the package is checked using an environment variable. This can be set using:

$export HIPS_EXTRA=\path\to\hips-extra

In Python, the path can be retrieved using the os module: os.environ['HIPS_EXTRA']. Now, what if the user does not have hips-extra repository …

Read more →