ASTRiDE uses an algorithm for quantifying the shape of each border to determine whether or not the border is a streak. ASTRiDE can detect not only long streaks but also relatively short or curved streaks.

1. Dependency
2. Installation
3. Test the Library
4. How to Use ASTRiDE
5. Test with Crowded Field Image
6. Application

• System Requirement
• ChangeLog
• Citation
• Contact

Python 2.7+ and 3.5+

Numpy 1.14+

• Numerical Python library.

Scipy 1.0.0+

• Scientific Python library.

Scikit-image 0.13.1+

• To get a contour map of a fits image.

Astropy 3.0+

• For reading a fits file.

Matplotlib 2.1.1+

• For plotting figures.

Phoutils 0.4+

• For calculating background map of a fits image.

These libraries will be automatically installed if your machine does not have them installed. If you encounter errors during the installation of these dependencies, try to install them individually. Your machine may not have other required libraries by these dependencies.

The easiest way to install the ASTRiDE package is:

pip install git+https://github.com/dwkim78/ASTRiDE

If you do not want to install/upgrade the dependencies, execute the above commend with the --no-deps option. ASTRiDE possibly works with older version of Python and other libraries.

Alternatively, you can download the ASTRiDE package from the Git repository as:

git clone https://github.com/dwkim78/ASTRiDE

cd ASTRiDE
python setup.py install

You can edit setup.py, if you do not want to update your own Python libraries (i.e. edit the install_requires variable).

Please see the AllskyTeam/allsky#84 (comment)

In short, 1) install pip, 2) install required packages, 3) install ASTRiDE, and 4) test ASTRiDE. It is tested on Raspberry Pi 4.

To check if ASTRiDE is correctly installed, type following commands in your Python console.

from astride import test

test()

The command will print messages like:

yyyy-mm-dd hh:mm:ss,mmm INFO - Start.
yyyy-mm-dd hh:mm:ss,mmm INFO - Read a fits file..
yyyy-mm-dd hh:mm:ss,mmm INFO - Search streaks..
yyyy-mm-dd hh:mm:ss,mmm INFO - Save figures and write outputs to ./long/
yyyy-mm-dd hh:mm:ss,mmm INFO - Done.

The test module will save figures and information of detected streaks under the "./long/" folder. In the folder, you can find two images and one text file. The two images are:

[ all.png ]

The output text file named as "streaks.txt" contains following information.

Most of these information are accessible using the ASTRiDE Streak instance as well. For details, see this section.

The following pseudo-code shows how to use ASTRiDE.

# Import the ASTRiDE library.
from astride import Streak

# Read a fits image and create a Streak instance.
streak = Streak('YOUR_FITS_FILENAME.fits')

# Detect streaks.
streak.detect()

# Write outputs and plot figures.
streak.write_outputs()
streak.plot_figures()

Detailed explanation for these commands are given in the following sections.

We first need to import the ASTRiDE library and create a Streak instance as:

from astride import Streak

streak = Streak('long.fits')

You can replace "long.fits" with your own fits filename. There are many options customizing the Streak instance such as:

Although you can customize pretty much everything of the Streak instance, it is recommended to leave them as they are until you understand each option. Some important options among these are explained through the following sections.

Now we can detect streaks in the fits image as:

streak.detect()

That's it! The above one-line command will do everything needed to detect streaks, which is:

• Background removal

  • ASTRiDE first removes background from the fits image. By default, ASTRiDE calculate background level and its standard deviation using sigma-clipped statistics (i.e. remove_bkg='constant'). If remove_bkg is 'map', the background map is derived using Phoutils. It calculates the map by sigma-clipping method within the box of the size "bkg_box_size".

• Contour map

  • Using the scikit-image, ASTRiDE derives the contour map of the fits image. The level of the contour is controlled by the "contour_threshold" value, such as: contour_threshold * background standard deviation (calculated when deriving the background map). Default "contour_threshold" is 3. The following images shows all the borders detected using the contour map.
  [ All the borders (color-coded) extracted using the contour map ]

• Streak determination based on the morphologies of each border
  • As we can see from the above figure, there are many borders of star-like sources that are definitely not streaks. ASTRiDE removes such star-like sources using morphological parameters derived from each border such as:

The following figure shows the remaining two streak after these cut.

[ Two streaks after the morphology cut. The numbers are their IDs. ]

• Link streaks by their slopes
  • As shown in the above image, ASTRiDE finally detected two streaks. However, these two streaks are not really separated two streaks. They seem to be one streak, but separately detected since the middle part of the streak is disconnected. ASTRiDE connects (i.e. link) such streaks by their slopes derived using the linear line fitting. If their slopes are within the "connectivity_angle", and also the slope between the two centers of the two streaks are within the "connectivity_angle" with each streak, ASTRiDE determines that the two streaks are connected. This is why the "all.png" shown in the section "Test" has only one red dashed-line box surrounding the two streaks. If one streak (i.e. s1) is determined to be linked with another streak (i.e. s2), s1's "connectivity" value is the index of s2. If s2 is again linked with s3, then again s2's "connectivity" is the index of s3. If s3 is not linked with any other streaks, s3's "connectivity" is -1.

Note that all the information derived during the above procedures are accessible using the Streak instance (See this section).

ASTRiDE provides functions to write outputs and plot figures as:

streak.write_outputs()

streak.plot_figures()

streak.write_outputs() will write an output text file, "streaks.txt", which is explained in the section "Test".

streak.plot_figures() will generate figures including "all.png" (shown in this section), and an individual figure for each linked streak. A filename of each individual file is the first index among the indices of the linked streaks such as "1.png" (shown below)

The Streak instance - after calling "detect()" function - contains many information such as:

Among these, streak.streaks contains a list of detected streaks. Each element has all the information that "streaks.txt" has (see this section). It also contains additional information such as:

Using the above information, you can plot your own figures.

The example shown above used a less-crowded field image. If there are many stars in the field (i.e. crowded field), it is possible that some stars' borders are attached to each other, which makes their borders long so that eventually look like a streak. In order to check how ASTRiDE works for such crowded field images, ASTRiDE is applied to a relatively crowded-field fits image. The following output images show the results.

[ Streak detection test using the crowded field image ]

These three images shown above are automatically generated by ASTRiDE. As you can see, ASTRiDE successfully excluded all the stars and detected two very short streaks (i.e. two bottom panels) that are quite hard to be detected even by eyes.

The following images show the application of ASTRiDE.

ESO DSS2 image	The Horsehead Nebular
[ Three streaks are detected. ]	[ No streak is detected. ]

In the case of the ESO DSS2 image (left panel), ASTRiDE detected three streaks. However, only one streak (i.e. ID=2, bottom center panel) looks like a real streak. The following table shows some parameters derived by ASTRiDE.

As we can clearly see, the streak with ID=2 is longer than two others, and also has a smaller shape factor than others. All these variables are accessible using the Streak instance, so you can define your own criteria according to the shapes of streaks that you want to detect. For instance, if you are interested in relatively long streaks, you can use the values of "Perimeter".

You can also set threshold cut for some of these parameters when creating the Streak instance. For details, see this section. For example, if you change the value of "shape_cut" to 0.3, ASTRiDE will also detect relatively thicker streaks. However, at the same time, it is also possible that ASTRiDE would return star-like objects.

In the case of the Horsehead Nebular image (right panel), it seems natural that ASTRiDE does not detect any streaks in the image.

The following example using the image of M35 (NGC 2168) gives an idea how to use the parameters mentioned above.

Default Setting	Parameter Modified

In the image, you can see the diffraction spike of the brightest stars, which is a sort of streaks. Left panel shows the result using the default ASTRiDE parameter setting, and it did not detect any streak. In the right panel, the two parameters were tuned (i.e. shape_cut = 0.3 and radius_dev_cut = 0.4), and ASTRiDE detected the spike. As you can clearly see, the spike shares its border with other stars, and it does not actually look like a streak anymore. That is the reason why it was not detected with the default setting. Nevertheless, as this example shows, you can decide your own criteria based on your needs.

Although ASTRiDE is not degisned to detect curves caused by gravitational lensing (e.g. see the following image), ASTRiDE can detects such curves as well.

[ Gravitational Lensing near SDSSCGB8842.3 and SDSSCGB8842.4 ]

As the figure shows, ASTRiDE detected four curved streaks around the galaxies. In this example, we tuned five parameters of ASTRiDE as follows:

Any decent or even relatively old machines can run ASTRiDE as long as the machines are capable of running general Python libraries. Runtime for streak detection (i.e. wall-clock time) varies according to the size of fit images and the crowdedness of the images. In the cases of the examples shown in the previous sections, it took from 0.6 seconds to 25 seconds using Macbook Pro 13'' equipped with 2.7 GHz Intel Core i5, 8 GB memory, and 256 GB SSD. If you want to decrease ASTRiDE runtime, increase the values of contour_threshold and min_points when creating the Streak instance. For details, see this section. Changing those values, however, could result in non-detection of short or faint streaks.

As you might notice, ASTRiDE does not use any source detection algorithm (e.g. Source Extractor) to distinguish stars from streaks. This is because such algorithms often find star-like-sources inside a streak. For instance, see the following figure.

[ Red circles are the sources detected by a source detection algorithm (i.e. DAOFIND-like algorithm) ]

Thus such source detection algorithms are not suitable to distinguish streaks from stars. One might think using the detected sources to construct streaks by somehow connecting them. Such methods, however, might not be very efficient either for 1) short streaks, or 2) crowded field.

If you want to write log messages either to console or to disk, you can use the ASTRiDE Logger class as:

from astride import Logger

logger = Logger().getLogger()

logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message')

Keep in mind that you need to generate only one logger instance through the whole processes, but not many. If you want to save log messages to a file, generate a logger instance as follows:

logger = Logger('/PATH/TO/FILE.log').getLogger()

This will send log messages to both console and a log file. Note that the path must be the absolute path.

• real-time training of an outlier model to detect streaks?
  • Tested with several different clustering algorithms (e.g. Birch, KMeans, hierarchical clustering, etc.), and it works well for clear outliers but not for ambiguous outliers (of course not since, strictly speaking, they are not even outliers). In other words, it detects long and thin streaks easily since they are clear outliers (i.e. they are not point sources), but for short and rather thick streaks, it fails to detect. In contrast, the current method using morphological parameters detects both kinds.
  • Nevertheless, it is possible to use clustering methods as supplementary detection methods for long streaks.

• Add additional information of detected streaks such as length, thickness, extreme points, etc.

• Fix minor bugs.

• Sky coordinates are also reported in RA, Dec

• fully_connected parameter of skimage.measure.find_contours is exposed to Streak class.
• fixed a bug when calling phoutils daofild.

• all Python library dependencies are updated to the latest versions.
• default output_path in Streak class is modified to include the full path of the input filepath.
• x/y plot coordinate swapping bug fixed.

• phoutils library update to v0.4.

• tested under Python 3.5

• phoutils library update to v0.3.

• add an option to choose which method to use for the background removal (i.e. remove_bkg option). See How to Use ASTRiDE for details.

• modify class/function comments so that it is more legible

• remove unnecessary image files for PyPi distribution

• bug fixed and source code cleaned

• beta version released

• initiate the GitHub repository

If you use this package for science publication, a citation to the ASCL entry would be appreciated, which is as follows:

@MISC{2016ascl.soft05009K,
       author = {{Kim}, Dae-Won},
        title = "{ASTRiDE: Automated Streak Detection for Astronomical Images}",
     keywords = {Software},
         year = 2016,
        month = may,
          eid = {ascl:1605.009},
        pages = {ascl:1605.009},
archivePrefix = {ascl},
       eprint = {1605.009},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2016ascl.soft05009K},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

Dae-Won Kim, email: dwkim78 at gmail.com

Webpage: https://sites.google.com/site/dwkim78/

astronomy - fits image - streak - satellite - meteor - NEO - fast-moving objects - boundary-tracing - contour-tracing - morphology