Skip to content
/ gems Public

Repository related to the manuscript "Automatic Gemstone Classification Using Computer Vision" by Bona Hiu Yan Chow and Constantino Carlos Reyes-Aldasoro published in Minerals MDPI, 2022, 12(1), 60 (doi: 10.3390/min12010060).

Notifications You must be signed in to change notification settings

hybchow/gems

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Automatic Gemstone Classification Using Computer Vision

Repository related to the manuscript "Automatic Gemstone Classification Using Computer Vision" by Bona Hiu Yan Chow and Constantino Carlos Reyes-Aldasoro published in the Minerals MDPI journal, 2022, 12(1), 60 (doi: 10.3390/min12010060).

Full-text PDF: https://www.mdpi.com/2075-163X/12/1/60/pdf

Abstract

This paper presents a computer-vision-based methodology for automatic image-based classification of 2042 training images and 284 unseen (test) images divided into 68 categories of gemstones. A series of feature extraction techniques (33 including colour histograms in the RGB, HSV and CIELAB space, local binary pattern, Haralick texture and grey-level co-occurrence matrix properties) were used in combination with different machine-learning algorithms (Logistic Regression, Linear Discriminant Analysis, K-Nearest Neighbour, Decision Tree, Random Forest, Naive Bayes and Support Vector Machine). Deep-learning classification with ResNet-18 and ResNet-50 was also investigated. The optimal combination was provided by a Random Forest algorithm with the RGB eight-bin colour histogram and local binary pattern features, with an accuracy of 69.4% on unseen images; the algorithms required 0.0165 s to process the 284 test images. These results were compared against three expert gemmologists with at least 5 years of experience in gemstone identification, who obtained accuracies between 42.6% and 66.9% and took 42–175 min to classify the test images. As expected, the human experts took much longer than the computer vision algorithms, which in addition provided, albeit marginal, higher accuracy. Although these experiments included a relatively low number of images, the superiority of computer vision over humans is in line with what has been reported in other areas of study, and it is encouraging to further explore the application in gemmology and related areas.

Dataset

A total of 2326 images of gemstones were obtained from Kaggle (accessed on 27 April 2021) and analysed in this work. The images were grouped into categories, and for this work the following 68 classes were selected for analysis: Alexandrite, Almandine, Amazonite, Amber, Amethyst, Ametrine, Andradite, Aquamarine, Aventurine Green, Aventurine Yellow, Benitoite, Beryl Golden, Bixbite, Bloodstone, Blue Lace Agate, Carnelian, Chalcedony, Chalcedony Blue, Chrome Diopside, Chrysoberyl, Chrysocolla, Chrysoprase, Citrine, Coral, Diamond, Diaspore, Dumortierite, Emerald, Fluorite, Hessonite, Iolite, Jasper, Kunzite, Kyanite, Lapis Lazuli, Malachite, Onyx Black, Onyx Green, Onyx Red, Peridot, Prehnite, Pyrite, Pyrope, Quartz Beer, Quartz Lemon, Quartz Rutilated, Quartz Smoky, Rhodochrosite, Rhodolite, Rhodonite, Ruby, Sapphire Blue, Sapphire Pink, Sapphire Purple, Sapphire Yellow, Serpentine, Sodalite, Spessartite, Sphene, Sunstone, Tanzanite, Tigers Eye, Topaz, Tourmaline, Tsavorite, Turquoise, Zircon and Zoisite. A total of 2042 images were used for training %the computer vision systems and 284 images were reserved for testing. For each class, 24--44 training images and 4--5 test images were available.

The original Kaggle dataset consists of 2856 images distributed in 87 classes, but some of these were discarded according to the following criteria.

  • The category Garnet was removed due to overlapping with Almandine, Pyrope, Rhodolite and Spessartine.

  • The Moonstone images displayed a variety of either orange, white or yellow colour, which was undesirable for the algorithms, and were thus eliminated.

  • Upon background segmentation, poorly segmented training images satisfying either of these conditions were discarded: (1) incomplete removal of background, or (2) extraction of only a minor portion of gemstone. Seventeen classes, namely Andalusite, Cats Eye, Danburite, Goshenite, Grossular, Hiddenite, Jade, Labradorite, Larimar, Morganite, Opal, Pearl, Quartz Rose, Scapolite, Spinel, Spodumene and Variscite, were removed, as fewer than 24 training images per class were retained.

  • test_masked contains the masked test images.

  • test_selected contains the selected test images.

  • train_masked contains the masked training images.

  • train_selected contains the selected training images.

Code files

All the algorithms used in this work were coded in Python 3.7.9.

Data Exploration and Pre-processing

  • InspectImages contains an initial inspection of the image dataset.

Background Segmentation

We applied Otsu thresholding based on grey-level intensity or the Saturation channel of HSV colour space to automatically extract the gemstones from the backgrounds.

Feature and algorithm comparison

Seven different machine-learning algorithms (Logistic Regression, Linear Discriminant Analysis, K-Nearest Neighbour, Decision Tree, Random Forest, Naïve Bayes and Support Vector Machine) were compared, each with 33 different feature extraction methodologies (including colour histograms, local binary pattern, Haralick texture and grey-level co-occurrence matrix properties), which provided a total of $ 7 \times 33 = 231$ combinations. Deep-learning classification (transfer learning) with ResNet-18 and ResNet-50 was also investigated. With the exception of transfer learning, the scripts were executed on a MacBook Pro equipped with a 2.3 GHz Intel Core i5 processor. Transfer learning was implemented on a virtual NVIDIA Tesla K80 Graphics Processing Unit (GPU) with two workers in Google Colaboratory.

  • KMeans-CIELAB.ipynb includes the extraction of CIELAB colour of the non-background K-means cluster centre.
  • KMeans-HSV.ipynb includes the extraction of HSV colour of the non-background K-means cluster centre.
  • KMeans-RGB.ipynb includes the extraction of RGB colour of the non-background K-means cluster centre.
  • MachineLearning.ipynb contains the extraction of all features except K-means clustering, as well as the training and evaluation of machine learning algorithms.
  • resnet.ipynb contains the implementation of transfer learning with ResNet-18 and ResNet-50.

Predictions by Expert Gemmologists

The expert group consisted of three gemmologists with both Graduate Gemologist of Gemological Institute of America (GIA) and Fellowship of the Gemmological Association of Great Britain qualifications and 5--8 years of experience in gemstone identification. The predictions of the test images made by each gemmologist are available in predictions1.csv, predictions2.csv and predictions3.csv.

Visualisations

Experimental findings were visualised using Python and Tableau. All figures used in the paper can be downloaded from the subfolder 'figures'.

Citation

If you find our work useful, please cite us as:

Chow, B.H.Y.; Reyes-Aldasoro, C.C. Automatic Gemstone Classification Using Computer Vision. Minerals 2022, 12, 60. https://doi.org/10.3390/min12010060

Chow BHY, Reyes-Aldasoro CC. Automatic Gemstone Classification Using Computer Vision. Minerals. 2022; 12(1):60. https://doi.org/10.3390/min12010060

Chow, Bona H.Y., and Constantino C. Reyes-Aldasoro. 2022. "Automatic Gemstone Classification Using Computer Vision" Minerals 12, no. 1: 60. https://doi.org/10.3390/min12010060

About

Repository related to the manuscript "Automatic Gemstone Classification Using Computer Vision" by Bona Hiu Yan Chow and Constantino Carlos Reyes-Aldasoro published in Minerals MDPI, 2022, 12(1), 60 (doi: 10.3390/min12010060).

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published