DISR: Dental Image Segmentation and Retrieval.

In this paper, we propose novel algorithms for retrieving dental images from databases by their contents. Based on special information of dental images, for better content-based dental image retrieval and representation, the image attributes are used. We propose Dental Image Segmentation and Retrieval (DISR), a content-based image retrieval method that is robust to translation and scaling of the objects in the images. A novel model is used to calculate the features of the image. We implemented the dentition plaster casts and proposed a special technique for segmenting teeth in our dental study models. For testing the efficiency of the presented algorithm, a software system is developed and 60 dental study models are used. The models are covering different kinds of malocclusions. Our experiments show that 95% of the extracted results are accurate and the presented algorithm is efficient.

INTRODUCTION

0The content-based image retrieval (CBIR) systems fall into two categories: the region-based and the image-based search. Some of the systems combine retrieval results from different algorithms with a weighted sum matching metric[1] or other merge techniques.[2] In a partial search querying system, a particular region in an image is searched.[3] Many efforts have been made for developing computerized systems in the area of clinical and dentistry applications. An automated expert system for orthodontics diagnosis is explained in.[45] Two clinical applications in the area of dentistry, based on computer-aided design and manufacturing, are described in.[67] One of such applications is implemented in,[6] in which an automated manufacturing system is introduced for dental filling in the area of inlays and crowns. A computer-assisted design for speeding up the production of dental restorations is proposed in,[8] which reduces the labor steps and provides better quality. Tooth rearrangement and a quantitative evaluation system in three-dimensional tooth movement for presurgery simulation developed in.[97] In this article, we present a robust and novel method to segment, store, retrieve and handle cases of mild malocclusion of the teeth dental study models.

In landmark-based methods section,, the landmark-based methods are described, which contains size of a specimen and the best landmarks as subsections. The Procrustes fitting and landmark-based shapes analyze is explained In procrustes fitting an landmark-based shapes analyze section and the dental study model is discussed in dental study models section. DISR algorithm section consists of nine subsections; the dental image segmentation and retrieval (DISR) algorithm is presented in this section. Retrieval evaluation method is explained in retrieval evaluation section and experimentation evaluation in experimentation evaluation section, and the conclusion is discussed in conclusion section.

LANDMARK-BASED METHODS

Landmarks are some numbers of annotated points that vary from one specimen to another. The proper landmarks are typically placed at the end points and sharp corners. Shape analysis landmarks method has some advantages compared with outline-based methods. We compare like with like and measure landmarks on all specimens, which is called homology. For ease of analysis, as less landmarks as possible are traced and coordinates are digitized in a complete outline. In addition to simple distance analysis between landmarks, many other complicated methods are available. From geometry point of views, the analyses of the results are sometimes difficult and sometimes easy to interpret.

The Size of a Specimen

The size of a specimen is deferent and depends on the measuring technique that is used for measuring the width, length, area or volume. One of the standard methods for measuring the size in landmarks is the centroid method, which can be reached in the following steps:

Find the centroid of landmarks. Centroid is the mean of all x and y coordinates of all the landmarks points

Sum up the squared distances of all landmarks from the centroid

Calculate the square root of the result.

The centroid size is calculated using Equation 1, where (x, y) are a set of landmarks and (x’, y’) is the centroid.

As an alternate method for measuring the size, we can calculate the square root of the area in a convex hull. The smallest possible convex polygon is called convex hull, such that all landmarks are either inside or on the vertices of the polygon [Figure 1].

Figure 1

Depiction of a convex hull and its landmark points

The Best Landmarks

Selecting the best suitable landmarks differs from one image to another. One of the simplest methods is using expert persons for annotating the images with a series of points, but this method is very time consuming and it is better to implement automated or semi-automated algorithms for annotation. In two-dimensional cases, the landmarks can be annotated at the corners and “T” junctions of the boundaries or easily located at biologically important landmarks; however, these types of points are very rare. They can be annotated with some more points along boundaries that are located at locations equally spaced from previously defined points [Figure 2]. A d-dimensional shape with n annotated points on it is represented by nd elements vector. For example, in two-dimensional image, the landmark points can be represented by {(xi, yi)}.

Figure 2

Landmarks at high curvature and junctions. The boundary points become more precise by intermediate points

PROCRUSTES FITTING AND LANDMARK-BASED SHAPES ANALYZE

One or a set of points are used as landmarks for viewing a shape, where the landmarks can be defined in two- or three-dimensional spaces. In thais regard, a wide and commonly used issue in the literature is called Procrustes analysis.[10] Procrustes fitting of two or a number of shapes moving, scaling and rotating the shapes the distances between shapes minimized, it is in non-Euclidean space, which makes it difficult to handle. Whenever a dental case has a small variation in shape, we use a Euclidean method and project the shape into a space that is called tangent space. However, this method is not precise, but tangent space more or less can coordinate a good approximation from the shape that is called Procrustes residuals, that is the difference between the mean shape and the Procrustes shapes. For calculating the Procrustes distance, first, we select two shapes, sum the squared Euclidian distances of the landmarks on shapes and then square root the result.

DENTAL STUDY MODELS

A special segmentation method for teeth dental study model is presented in this article, which can detect several types of malocclusions. A malocclusion is a misalignment of teeth or incorrect relation between the teeth of two dental arches. The cuspid tooth is perhaps the most recognizable tooth in the mouth because of its pointed shape and length. Located between the incisors and first premolar, cuspids are primarily used for griping and tearing food. Because the cuspid is the longest tooth with one firmly implanted root, it helps guide the teeth and jaws into their proper biting position. Human adults will develop four cuspids, one in each dental quadrant, sometime between ages 11 and 12. Next to the wisdom teeth, cuspids may become impacted in the bone.

Watershed method is used for locating orthodontic features such as tips of the cuspids, cutting edges of the incisors and cusps of the premolars and molars.[45] Both the systems in[45] are designed for distinguishing the mild malocclusion; in the proposed system in this article we targeted the same. The teeth arrangement comes in different shapes from one dental case to another, and it makes the segmentation algorithm difficult. The difficulty arises mostly because of misalignments in the teeth.

DISR ALGORITHM

Whenever the form of the dental arch in orthodontics investigation is essential,[11] the degree of malocclusion is needed to be calculated; therefore, orthodontists need to know about maxillary and mandibular arches.[5] One of the most important roles in our presented tooth segmentation algorithm is the dental arches. In the DISR computerize method, the dental arch is detected in a simple process, which starts with manually marking several points in the image by the user[12] or interactively defined arch as described in.[1314] We improved the mentioned methods of[511-14] and proposed a novel method for detecting the tooth-based dental arch.

Marking the Landmarks

The experts are used to annotate the dental study model with three types of points. For each tooth, one point at the centroid point and another on the contour of the Buccal side [Figure 3]. The third type of landmarks is the one that is annotated at the contact point of the adjacent teeth [Figure 3]. The contour point is located at the middle of the buccal side and at the contour of a tooth. Two adjacent teeth may have several contact points, but only the closest one to the buccal side is selected.

Figure 3

Three types of landmarks and depict of the inspection spoke rotated in two different directions

Inspection Spoke

The inspection spokes are fitted by experts such that they divide the angle between two tangent lines of neighboring teeth into half [A1=A2]. In Figure 4, four samples with different rotations are shown. In all of these samples, the inspection spoke divides the tangent lines into two equal angles [A1=A2].

Figure 4

The inspection spoke divides the angle of the two tangent lines into half [A1=A2]

Data Acquisition and Storage

Several methods are reported in the literature that are used for acquiring digital data. The depth-from-absorption method is used for obtaining the three-dimensional profile of wax imprints as stated in,[45] dental study models are scanned by range scanner and the produced images are displayed and stored the same as the[1516] methods. Because of what is stated above, the landmark points are annotated and inspection spokes are fitted into dental study models by experts. The dental study models are scanned by a commercial laser scanner and digital images are provided. The laser light is projected vertically onto the plaster cast while it has been placed on its platform. The scanned images from 60 dental study models are stored in a database and used as an input data file for testing the performance of the DISR's software system.

The Teeth Alignment Lines

The DISR software system draws a tooth alignment line for each tooth. As shown in Figure 5 for each tooth, the alignment line goes through the contour point and center point landmarks.

Figure 5

The teeth alignment lines A and C and the inspection spoke B

Produced Images for Each Orthodontic Case

The DISR system produces four images for each of the orthodontic cases, one from the segmented teeth of the maxillary plan view [Figure 6a] and another from the panoramic view of the same cast [Figure 6b].

Figure 6

Four types of images produced by the dental image segmentation and retrieval system. (a) A plan view of class I, the bi-maxillary proclination with segmented maxilla; (b) Panoramic view of the same cast; (c) Plan view of class II, division 1 with segmented mandible; (d) Panoramic view of the same cast

The third and fourth produced images are from the mandible plan view and panoramic view of the same cast [Figure 6c and d]. For producing the panoramic images [Figure 6b and d], first the buccal sides of the casts are marked with points by an expert and then scanned with a scanner into the memory. The scanned images from two buccal side views are attached by the software system and produce a panoramic image. The alignment lines shown in Figure 6 are added by the DISR software system after the landmarked casts are scanned into the memory.

Curve Fitting

For each orthodontic case, eight curves are created by the DISR system [Figure 7].

Figure 7

Contour curve, center curve, inspection spokes and alignment lines

Curve fitted through the center points of the maxillary teeth

Curve fitted through the contour points of the maxillary teeth

Curve fitted through the center points of the mandible teeth

Curve fitted through the contour points of the mandible teeth

Curve fitted through the center points of the panoramic view of the maxillary teeth

Curve fitted through the contour points of the panoramic view of the maxillary teeth

Curve fitted through the center points of the panoramic view of the mandible teeth

Curve fitted through the contour points of the panoramic view of mandible teeth. Let us name the curve that goes through the center points as “center curve” and the one that goes through contour points as “contour curve.”

Rotation Angles

The DISR system detects six different types of crosses for calculating the rotation angles [Figure 8]. The calculated angles are classified into the following types:

Figure 8

Six types of rotation angles t1…t6

t1: If an alignment line crosses the previous inspection spoke on the buccal side

t2: If an inspection spoke crosses the previous inspection spoke on the lingual side

t3: If an alignment line crosses the next inspection spoke on the buccal side

t4: If an alignment line crosses the previous inspection spoke on the lingual side

t5: If an inspection spoke crosses the previous inspection spoke on the buccal side

t6: If an alignment line crosses the next inspection spoke on the lingual side.

In orthodontic therapies, the rotation angles are needed. The proposed software system calculates the t1…t6 angles of the stated images [Figure 8] for maxillary and mandible teeth.

Created Databases

The databases that created and used by DISR are as follows:

The original scanned images from casts without added landmarks and spokes, stored in database DB-ORIGINAL.

The scanned images from casts with added landmarks and spokes, stored in database DB-LANDMARED.

For an orthodontic case with 14 teeth in the upper jaw and 14 teeth in the lower jaw (i.e., Figure 6).

The annotated points are as follows:

14 center points on maxillary teeth

14 contour points on maxillary teeth

14 center points on mandible teeth

14 contour points on mandible teeth

14 center points on the panoramic view of maxillary teeth

14 contour points on the panoramic view of maxillary teeth

14 center points on the panoramic view of mandible teeth

14 contour points on the panoramic view of mandible teeth

14 contact points on maxillary teeth

14 contact points on mandible teeth

13 contact points on panoramic view of maxillary teeth

13 contact points on panoramic view of mandible teeth.

Therefore, for each orthodontic case, the calculated rotation angles (t1…t6, if there is any) and also XY-coordinate of 164 above-stated points stored as features of a feature vector in database DB-FEATURES. The feature vectors are indexed based on teeth attributes and their locations on the upper or lower jaws.

Image Retrieval and Malocclusions Reporting

Similar images can be retrieved from databases using query by example method. For any given query cast, first, the landmark points are annotated and spokes fitted in the cast by an expert and then the cast is scanned by a scanner, alignment lines are added to the images by the system, rotation angles are calculated by the system and, finally, the feature vector is prepared for query cast. With respect to attributes of the feature vector, the DISR system retrieves similar cases. The user can define the similarity thresholds.

DISR draws the entire eight center and contour curves [curve fitting subsection] for query. Using the Procrustes method [procrustes fitting and landmark-based shapes analyze section, equation 3], the system depicts best fitting results from the query and retrieved cases. The orthodontic experts select the closest case or the one that is close enough to the query case. Besides displaying the results from Procrustes fitting, this system is designed to report cases of mild malocclusion.

RETRIEVAL EVALUATION

We use the same method as in[17] for evaluating the performance of our proposed method.

Equation 2 is used for ranking part of the top r ranked images, which are similar to query image. In this equation, P is the precision and r is the cutoff point.

In Equation 3, Rr is the recall and r is the proportion of the most similar images that were retrieved.

Achieving a better and higher recall normally results in a lower precision and finally ends with retrieving the entire images. An optimized precision normally results with smaller recall and finally at the bottom end retrieves only one or two images. The precision estimating is much easier than recall; an expert can simply scan the images and determine which one is most relevant to the query.

For estimating the recall value, if the collection is small, it is possible that an expert scans the entire collection to determine the set of similar images, but, in a large collection sometimes, it is very difficult to determine such entire collection. There are varieties of systems that can be used for estimating the relevant images in the answers. In a set of N test queries, to evaluate the performance level, for each recall level r, we average precision levels [Equation 4].

The recall and precision curves can be used for evaluating and comparing two systems and, usually, partition the recall levels into 11 standard levels: from 0% up to 100%.

EXPERIMENTATION EVALUATION

For evaluation of information retrieval, a standard method is used by the CBIR community, which is called Precision and Recall (PR) method.[18] The produced graph by this method carry much information.

The PR graph can present useful and detailed information in a region or even mislead the user because of poor performance of some omitted areas. The evaluation of the performance based on a partial graph is very difficult, and it is better to be used with the complete graph simultaneously. In this experiment, we implemented a small but relatively complete database that contains images of 60 orthodontic cases. The DISR ran with four different query images, which we classified into very hard, hard, easy and very easy query images. In Figure 9, we can see how the PR graphs distinct between different results. As shown in Figure 9, the very hard query curve decreases much faster and starts later than the hard query one. Compared with hard and very hard curves, the easy and very easy curves are much later and their behaviors are alternative, but, in average, these two curves look almost alike.

Figure 9

Precision versus recall graph for four different query cases

To evaluate performance, we calculate the average precision levels [Equation 6] over a set of four test queries at each recall level:

Average precision for very easy query = (1 + 0.97 + 0.88 + 0.50 + 0.40 + 0.35) / 6 = 68%

Average precision for easy query = (1 + 0.85 + 0.85 + 0.60 + 0.35 + 0.28) / 6 = 66%

Average precision for hard query = (1 + 0.55 + 0.40 + 0.30 + 0.27 + 0.24) / 6 = 46%

Average precision for very hard query = (1 + 0.50 + 0.29 + 0.27 + 0.21 + 0.21) / 6 = 41%

As can be seen, the value of average precision decreases as much as the query becomes harder.

CONCLUSION

In this paper, we present novel algorithms for retrieval of dental images by content. We propose the DISR, a new content-based image retrieval method that is robust to translation and scaling of the objects in images. We presented a special teeth segmentation method based on dental plaster casts as implemented models. For testing the efficiency and robustness of the presented system, 60 dental study models are used. The models represent different kinds of malocclusions. Our experimental results show a more than 95% accuracy in detecting malocclusions in the images, which is satisfactory. The presented software system can be implemented by the dentist for automatic storing and retrieval of malocclusions cases of the patients for their future references, which will help the clinician to maximize the diagnostic acumen, treatment planning process and proper execution of appropriate treatment strategy for confident clinical problem-solving.

BIOGRAPHY

Abdol Hamid Pilevar, Assistant Professor in Computers Engineering Department, Bu Ali Sina University, Hamedan, Iran. He received his BSc and MSc degrees in Computer Systems from Florida Atlantic University, FL, USA.

Dr. Pilevar received his PhD degree in Computer Science from University of Mysore, Mysore, India in 2005

He was a Research Associate and Post Doctoral Fellow at the Indian Institute of Science (IISC, Bangalore, India) and Mediscan Prenatal Diagnosis & Fetal Therapy Center (Chennai, India) between 2005 and 2006

His fields of interest include Medical Intelligence and Image Processing, 3D Modeling and Speech and Natural Language Processing. He has published more than 50 papers in international and national journals and conferences