Abstract. 

Image registration is the alignment (i.e. finding a general coordinate system) of two or more images of the same scene. Complex case of this task is the multimodal image registration, for example, optical-to-SAR. The need for such registration appeared in image fusion and object detection. The research on optical-to-SAR image registration shows that there are no publicly available datasets with the enough size for debugging and testing algorithms. In many works, testing is carried out on a several pairs of images that are not always publicly available. Moreover, different works use different datasets. In this paper we present a dataset of 100 aligned optical-SAR pairs of images. Additionally, we consider methods for evaluating keypoints repeatability, the accuracy of their matching, and a method for calculating the accuracy of image registration for optical-SAR data. Using these methods, we compare the results of these methods for the classical SIFT, YAPE and Harris detectors and SIFT, ORB and SURF descriptors on the presented dataset.

Keywords: 

image registration, repeatability of keypoints.

PP. 44-57.

DOI 10.14357/20718632210205

 

References

1. Sidorchuk D. S., Volkov V. V. Kompleksirovanie radiolokacionnyh izobrazhenij i opticheskih snimkov v vidimom i teplovom diapazonah s uchetom razlichij v vospriyatii yarkosti i cvetnosti [Fusion of radar, visible and thermal imagery with account for differences in brightness and chromaticity perception]. // Sensornye sistemy [Sensory systems]. 2018. V. 32. № 1. P. 14–18. DOI: 10.7868/S0235009218010031.

2. Errico A., Angelino C. V., Cicala L., Persechino G., Ferrara C., Lega M., Vallario A., Parente C., Masi G., Gaetano R., Scarpa G. Detection of environmental hazards through the feature-based fusion of optical and SAR data: A case study in southern Italy. // International Journal of Remote Sensing. 2015. V. 36. № 13. P. 3345–3367. DOI: 10.1080/01431161.2015.1054960.

3. Plank S., Twele A., Martinis S. Landslide mapping in vegetated areas using change detection based on optical and polarimetric SAR data. // Remote Sensing. 2016. V. 8. № 4. P. 307. DOI: 10.3390/rs8040307.

4. Ye S. P., Chen C. X., Nedzved A., Jiang J. Building detection by local region features in SAR images. // Computer Optics. 2020. V. 44. № 6. P. 944–950. DOI: 10.18287/2412-6179-CO-703.

5. Shi W., Su F., Wang R., Fan J. A visual circle based image registration algorithm for optical and SAR imagery. // In 2012 IEEE International Geoscience and Remote Sensing Symposium. 2012. P. 2109–2112.

6. Suri S., Reinartz P. Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. // IEEE Transactions on Geoscience and Remote Sensing. 2009. V. 48. № 2. P. 939–949. DOI 10.1109/TGRS.2009.2034842.

7. Gong M., Zhao S., Jiao L., Tian D., Wang S. A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. // IEEE Transactions on Geoscience and Remote Sensing. 2013. V. 52. № 7. P. 4328–4338. DOI: 10.1109/TGRS.2013.2281391.

8. Fan B., Huo C., Pan C., Kong Q. Registration of optical and SAR satellite images by exploring the spatial relationship of the improved SIFT. // IEEE Geoscience and Remote Sensing Letters. 2012. V. 10. № 4. P. 657–661. DOI: 10.1109/LGRS.2012.2216500.

9. Ma W., Wen Z., Wu Y., Jiao L., Gong M., Zheng Y., Liu L. Remote sensing image registration with modified SIFT and enhanced feature matching. // IEEE Geoscience and Remote Sensing Letters. 2016. V. 14. № 1. P. 3–7. DOI: 10.1109/LGRS.2016.2600858.

10. Paul S., Pati U. C. Optical-to-SAR image registration using modified distinctive order based self-similarity operator. // In 2018 IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS). 2018. P. 1–5.

11. Xiang Y., Wang F., You H. OS-SIFT: A robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas. // IEEE Transactions on Geoscience and Remote Sensing. 2018. V. 56. № 6. P. 3078– 3090. DOI: 10.1109/TGRS.2018.2790483.

12. Paul S., Pati U. C. Automatic optical-to-SAR image registration using a structural descriptor. // IET Image Processing. 2019. V. 14. № 1. P. 62–73. DOI: 10.1049/ietipr. 2019.0389.

13. Xiong X., Xu Q., Jin G., Zhang H., Gao X. Rank-Based Local Self-Similarity Descriptor for Optical-to-SAR Image Matching. // IEEE Geoscience and Remote Sensing Letters. 2019. V. 17. № 10. P. 1742–1746. DOI: 10.1109/LGRS.2019.2955153.

14. Wang H., Wang C., Li P., Chen Z., Cheng M., Luo L., Liu Y. Optical-to-SAR Image Registration Based On Gaussian Mixture Model. // ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2012. V. 39. P. 179–183.

15. Kunina I., Panfilova E., Gladkov A. Matching of SAR and optical images by independent referencing to vector map. // In Eleventh International Conference on Machine Vision (ICMV 2018). 2019. V. 11041. DOI: 10.1117/12.2523132.

16. Fischler M.A., Bolles R.C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. // Communications of the ACM. 1981. V. 24. № 6. P. 381–395.

17. Chekanov M.O., Shipitko O.S., Ershov E.I. Odnotochechnyj RANSAC dlya ocenki velichiny osevogo vrashcheniya ob"ekta po tomograficheskim proekciyam [1-point RANSAC for axial rotation angle estimation by tomographic projections]. // Sensornye sistemy [Sensory systems]. 2020. V. 34. № 1. P. 72–86. DOI: 10.31857/S0235009220010060. 

18. Tropin D. V., Nikolaev D. P., Slugin D. G. The method of image alignment based on sharpness maximization. // In Eleventh International Conference on Machine Vision (ICMV 2018). 2019. V. 11041. DOI: 10.1117/12.2522903.

19. Shemiakina J.A. Ispol'zovanie tochek i pryamyh dlya vychisleniya proektivnogo preobrazovaniya po dvum izobrazheniyam ploskogo ob"ekta [Using points and straight lines to calculate a projective transformation from two images of a flat object]. // Informacionnye tekhnologii i vychislitel'nye sistemy [Journal of Information Technologies and Computing Systems]. 2017. V. 3. P. 79–91.

20. Skoryukina N., Faradjev I., Bulatov K., Arlazarov V.V. Impact of geometrical restrictions in RANSAC sampling on the ID document classification. // In Twelfth International Conference on Machine Vision (ICMV 2019). 2020. V. 11433. DOI: 10.1117/12.2559306.

21. Xu C., Sui H., Li H., Liu J. An automatic optical and SAR image registration method with iterative level set segmentation and SIFT. // International Journal of Remote Sensing. 2015. V. 36. № 15. P. 3997–4017. DOI: 10.1080/01431161.2015.1070321.

22. Tropin D.V., Shemiakina J.A., Konovalenko I.A., Faradjev I.A. O lokalizacii ploskih ob"ektov na izobrazheniyah so slozhnoj strukturoj proektivnyh iskazhenij [Localization of planar objects on the images with complex structure of projective distortion]. // Informacionnye processy [Information processes]. 2019. V. 19. № 2. P. 208–229.

23. Optical-SAR dataset and method for evaluating image registration algorithms. Available at:

https://github.com/VolkovVl/Optical-SAR-dataset (accessed April 9, 2021).

24. Lowe D.G. Distinctive image features from scale-invariant keypoints. // Int. J. Comput. Vis. 2004. V. 60. № 2. P. 91–110.

25. Lepetit V., Fua P. Towards Recognizing Feature Points using Classification Trees. // Technical report, Swiss Federal Institute of Technology (EPFL). 2004.

26. Sedaghat A., Mokhtarzade M., Ebadi H. Uniform robust scale-invariant feature matching for optical remote sensing images. // IEEE Transactions on Geoscience and Remote Sensing. 2011. V. 49. № 11. P. 4516–4527.

27. Zhao D., Yang Y., Ji Z., Hu X. Rapid multimodality registration based on MM-SURF. // Neurocomputing. 2014. V. 131. P. 87–97. DOI: 10.1016/j.neucom.2013.10.037.

28. Xiang Y., Tao R., Wang F., You H. Automatic Registration of Optical and SAR Images VIA Improved Phase Congruency. // In IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium. 2019. P. 931–934.

29. Ye Y., Shen L., Hao M., Wang J., Xu Z. Robust optical-to-SAR image matching based on shape properties. // IEEE Geoscience and Remote Sensing Letters. 2017. V. 14. № 4. P. 564–568. DOI: 10.1109/LGRS.2017.2660067.

30. Ye Y., Shan J., Bruzzone L., Shen L. Robust registration of multimodal remote sensing images based on structural similarity. // IEEE Transactions on Geoscience and Remote Sensing. 2017. V. 55. № 5. P. 2941–2958. DOI: 10.1109/TGRS.2017.2656380.

31. Copernicus Open Access Hub, Terms and Conditions. Available at:

https://scihub.copernicus.eu/twiki/do/view/SciHubWebPor

tal/TermsConditions (accessed January 22, 2021).

32. Copernicus Open Access Hub. Available at:

https://scihub.copernicus.eu/ (accessed December 29, 2020).

33. OpenCV: Main page. Available at:

https://docs.opencv.org/master/index.html (accessed April 06, 2021).