このページは http://www.slideshare.net/EngHaitham/comparison-of-image-segmentation の内容を掲載しています。

掲載を希望されないスライド著者の方は、こちらよりご連絡下さい。

1年以上前 (2015/02/19)にアップロードin学び

An evaluation of two popular segmentation algorithms, the mean shift-based segmentation algorithm...

An evaluation of two popular segmentation algorithms, the mean shift-based segmentation algorithm and a graph-based segmentation scheme. We also consider a hybrid method which combines the other two methods.

- A Comparison of Image Segmentation

Algorithms

Presented by

Haitham Abdel-atty

Abdullah

Yara Bahaa El-Din Hashem

Pre-Masters 2014-2015

Supervised by:

Prof. Dr. Mostafa Gadal-

Haqq

1 - Agenda

Introduction

Image Segmentation Algorithms

› Mean Shift Segmentation

› Efficient Graph-based Segmentation

› Hybrid Segmentation Algorithm

Normalized Probabilistic Rand (NPR)

Index

Experiments

Conclusion

2 - Introduction

Image segmentation

Is the process of partitioning a digital image into

multiple segments (sets of pixels)

The goal of segmentation

Is to simplify and/or change the representation of

an image into something that is more meaningful

and easier to analyze

3 - Introduction (Cont.)

We will present an evaluation of two popular

segmentation algorithms, the mean shift-based

segmentation algorithm and a graph-based

segmentation scheme. We also consider a hybrid

method which combines the other two methods.

we compare all use the same image features (position

and color) for segmentation, thereby making their

outputs directly comparable.

4 - Introduction (Cont.)

For each of these algorithms, we examine three characteristics:

1. Correctness: the ability to produce results that are consistent

with ground truth

2. Stability with respect to parameter choice: the ability to

produce segmentations of consistent correctness for a range of

parameter choices.

3. Stability with respect to image choice: the ability to

produce segmentations of consistent correctness using the same

parameter choice on a wide range of different images.

The Normalized Probabilistic Rand (NPR) index is used to measure the

above characteristics.

5 - Mean shift

6 - The mean shift algorithm

Is a nonparametric clustering technique which does not

require prior knowledge of the number of clusters, and

does not constrain the shape of the clusters.

Mean shift is used for image segmentation, clustering,

visual tracking, space analysis, mode seeking ...

Technique for clustering-based segmentation

7 - The mean shift algorithm (Cont.)

The key to mean shift is a technique for efficiently finding

peaks (highest density or mode) in this high-dimensional

data distribution

8 - The mean shift algorithm (Cont.)

Density Estimation

Discrete PDF Representation

(PDF : probability density function)

Data

Gradient Estimation

(Mean Shift)

PDF Analysis

9 - Density Estimation

1 n

P(x)

K (x - x )

Kernel Density Estimation is a function of some

i

n i 1

finite number of data points x …x

1

n

Data

Assumed Underlying PDF

Real Data Sample

10s - Gradient Estimation (Mean Shift)

Region of

interest

Center of

mass

Mean Shift

vector

11 - Gradient Estimation (Mean Shift)

Region of

interest

Center of

mass

Mean Shift

vector

12 - Gradient Estimation (Mean Shift)

Region of

interest

Center of

mass

Mean Shift

vector

13 - Gradient Estimation (Mean Shift)

Region of

interest

Center of

mass

Mean Shift

vector

14 - Gradient Estimation (Mean Shift)

Region of

interest

Center of

mass

Mean Shift

vector

15 - Gradient Estimation (Mean Shift)

Region of

interest

Center of

mass

Mean Shift

vector

16 - Gradient Estimation (Mean Shift)

Region of

interest

Center of

mass

17 - Gradient Estimation (Mean Shift)

18 - Gradient Estimation (Mean Shift)

Simple Mean Shift procedure:

• Compute mean shift vector

•Translate the Kernel window by m(x)

2

n

x - x

i

x g

i

h

i 1

m(x)

x

2

n

x - x

i

g

h

i 1

19 - Real Modality Analysis

20 - Attraction basin

Attraction basin: the region for which all

trajectories lead to the same peak (mode)

Cluster: all data points in the attraction basin of

a mode

21 - Attraction basin

22 - Mean shift segmentation results

23 - Mean shift pros and cons

Pros

Does not assume spherical clusters

Just a single parameter (window size)

Robust to outliers

Cons

Computationally expensive.

Have to choose kernel size in advance

Output depends on window size.

Not suitable for high-dimensional features.

24 - Efficient Graph-based

Segmentation

25 - Efficient Graph-based Segmentation

Another method of performing clustering

in feature space.

Works on data points in feature space

without first performing a filtering step.

Key to success of this method is adaptive

thresholding.

26 - Efficient Graph-based

Segmentation (Cont.)

Represent features and their relationships

using a graph

Manipulate the graph to segment the

image

27 - Images as graphs

j

wij

i

Node for every pixel

Edge between every pair of pixels (or every

pair of “sufficiently close” pixels)

Each edge is weighted by the similarity of the

two nodes

28 - Segmentation by graph partitioning

j

wij

i

A

B

C

Break Graph into Segments

› Delete links that cross between segments

› Easiest to break links that have low affinity

similar pixels should be in the same segments

dissimilar pixels should be in different segments

29 - Measuring Affinity

Distance

aff x

, y exp 1

2 2 x y 2

d

Intensity

aff x

, y exp 1

2

2 2 I x

I y

i

Color

aff x

, y exp 1

2

2 2 c x

c y

t

30 - Scale affects affinity

Small σ: group only nearby points

Large σ: group far-away points

31 - Changing scores for different parameters using efficient

graph-based segmentation: (a) Original image, (b)-(d) efficient

graph-based segmentations using scale bandwidth (h ) 7, color

s

bandwidth (h ) 7 and k values 5, 25, and 125 respectively.

r

33 - Hybrid Segmentation

34 - Hybrid Segmentation

Combine two previous methods

we apply mean shift filtering, and then we

use efficient graph-based clustering to

give the final segmentation.

The quality of the segmentation is high.

35 - Example of changing scores for different parameters using a hybrid

segmentation algorithm which first performs mean shift filtering and then

efficient graph-based segmentation: (a) Original image, (b)-(g)

segmentations using scale bandwidth (h ) 7, and color bandwidth (h ) and k

s

r

value combinations (3,5), (3,25), (3,125), (15,5), (15,25), (15,125)

respectively.

36 - Normalized Probabilistic

Rand (NPR) Index

37 - Normalized Probabilistic Rand (NPR)

Index

X

The Rand index (RI) or Rand measure

(named after William M. Rand) is a measure of the

similarity between two data clustering.

a d

RI(P,G) a b c d

G

P

a

a

a

a

b

d

b

d

c

c

The Rand index has a value between 0 and 1.

38 - Normalized Probabilistic Rand (NPR)

Index (Cont.)

The Rand index (RI)

a

a d

RI(P,G) a b c d

RI(P,G)

a + b + c + d

b

39 - Normalized Probabilistic Rand (NPR)

Index (Cont.)

The Probabilistic Rand Index (PRI)

counts the fraction of pairs of pixels whose labels are consistent between

the computed segmentation and the ground truth, averaging across

multiple ground truth (manual) segmentations to account for scale

variation in human perception.

In other simple words, PRI measuring the similarity between two

partitions.

40 - Normalized Probabilistic Rand (NPR)

Index (Cont.)

In PRI agreements ( ) and disagreements ( ) at

the pixel-pair are weighted according to the probability of their

occurring.

Multiple ground truth (manual)

segmentations

Computed segmentation

41 - Normalized Probabilistic Rand (NPR)

Index (Cont.)

The PR index does however have one serious flaw. Note that the PR index is

on a scale of 0-1, but there is no expected value for a given segmentation.

That is, it is impossible to know if any given score is good or bad.

The significance of a measure of similarity has much to do with the baseline

with respect to which it is expressed.

For image segmentation, the baseline may be interpreted as the expected

value of the index.

All of these issues are resolved with normalization to produce the Normalized

Probabilistic Rand (NPR) index

PRI

Baseline

NPR Index

Is one

42 - Experiments

43 - Experiments

‘EDISON’ refers EDISON system for mean

shift segmentation.

‘FH’ refers to the efficient graph-based

segmentation method.

‘MS+FH’ refers to our hybrid algorithm of

mean shift filtering followed by efficient

graph-based segmentation.

All of the experiments were performed on

the publicly available Berkeley image

segmentation database which contains 300

images of natural scenes.

44 - Examples of images from the Berkeley image

segmentation database

45 - Experiments (Cont.)

we will divide each dimension by the

corresponding {hs, hr} as in the EDISON

system. So each algorithm was run with a

parameter combination from the sets:

hs = 7,

hr = {3, 7, 11, 15, 19, 23}, and

k = {5, 25, 50, 75, 100, 125}.

46 - Maximum performance

Maximum NPR indices achieved on individual images with the set of

parameters used for each algorithm. Plot (a) shows the indices achieved

on each image individually, ordered by increasing index. Plot (b) shows

the same information in the form of a histogram.

47 - Maximum performance

All of the algorithms produce similar

maximum NPR indices, demonstrating that

they have roughly equal ability to produce

correct segmentations with the parameter

set chosen.

Few images which have below-zero

maximum NPR index.

48 - Average performance per

image

An algorithm which creates good

segmentations will have a histogram skewed

to the right.

A standard deviation histogram that is

skewed to the left indicates that the

algorithm in question is less sensitive to

changes in its parameters.

Using the means as a measure certainly makes

us more dependent on our choice of

parameters for each algorithm.

49 - Average performance per

image (Cont.)

Average performance over all parameter

combinations:

› Mean NPR plots for each of the three systems

with averages taken over all possible

combinations of the parameters h and k

r

50 - 51
- Mean NPR indices achieved using each of the segmentation algorithms. The

first row shows results from the mean shift-based system (EDISON), the

second from the efficient graph-based system (FH), and the third from the

hybrid segmentation system (MS+FH). Results from each algorithm are given

for individual images over the parameter set of all combinations of hr

= {3, 7, 11, 15, 19, 23} and k = {5, 25, 50, 75, 100, 125}. Plots (a), (d) and (g)

show the mean indices achieved on each image individually, ordered by

increasing index, along with one standard deviation. Plots (b), (e) and (h)

show histograms of the means. Plots (c), (f) and (i) show histograms of the

standard deviations.

52 - Average performance per

image (Cont.)

Average performance over different

values of the color bandwidth h :

r

› NPR indices averaged over values of h , with k

r

held constant

53 - 54
- Mean NPR indices achieved using the efficient graph-based

segmentation system (FH) on individual images over the parameter set

h = {3, 7, 11, 15, 19, 23} with a constant k. Plot (a) shows the mean

r

indices achieved on each image individually, ordered by increasing

index, along with one standard deviation. Plot (b) shows a histogram of

the means. Plot (c) shows a histogram of the standard deviations.

55 - 56
- Mean NPR indices achieved using the hybrid segmentation system

(MS+FH) on individual images over the parameter set h = {3, 7, 11, 15,

r

19, 23} with a constant k. Plot (a) shows the mean indices achieved on

each image individually, ordered by increasing index, along with one

standard deviation. Plot (b) shows a histogram

of the means. Plot (c) shows a histogram of the standard deviations.

57 - Average performance per

image (Cont.)

Average performance over different

values of k

› Mean NPR indices as k is varied and h is

r

held constant.

58 - 59
- Mean NPR indices achieved using the efficient graph-based segmentation

system (FH) on individual images over the parameter set k = {5, 25, 50,

75, 100, 125} with a constant hr. Plots (a), (d) and (g) show the mean

indices achieved on each image

individually, ordered by increasing index, along with one standard

deviation. Plots (b), (e) and (h) show histograms of the means. Plots (c), (f)

and (i) show histograms of the standard deviations.

60 - 61
- Mean NPR indices achieved using the hybrid segmentation system

(MS+FH) on individual images over the parameter set k = {5, 25, 50, 75,

100, 125} with a constant hr. Plots (a), (d) and (g) show the mean

indices achieved on each image individually, ordered by increasing

index, along with one standard deviation. Plots

(b), (e) and (h) show histograms of the means. Plots (c), (f) and (i) show

histograms of the standard deviations.

62 - Average performance per

parameter choice

The final set of experiments looks at the

stability of a particular parameter

combination across images.

In each experiment results are shown with

respect to a particular parameter, with

averages and standard deviations taken

over segmentations of each image in the

entire image database.

63 - Average performance per

parameter choice (Cont.)

Average performance over all images for

different values of h :

r

Mean NPR indices using the

EDISON segmentation system on

each color bandwidth (h ) over the

r

set of images, with one standard

deviation.

64 - Mean NPR indices using graph-based segmentation (FH) on

each color bandwidth h = {3, 7, 11, 15, 19, 23} over the set of

r

images. One plot per value of k.

65 - Mean NPR indices using hybrid segmentation (MS+FH) on

each color bandwidth h = {3, 7, 11, 15, 19, 23} over the

r

set of images. One plot per value of k.

66 - Average performance per

parameter choice (Cont.)

Average performance over all images for

different values of k

› Examine the stability of k over a set of images.

67 - Mean NPR indices using efficient graph-based

segmentation (FH) on each of k = {5, 25, 50, 75, 100,

125} over the set of images. One plot per value of h .r

68 - Mean NPR indices using hybrid segmentation

(MS+FH) on each of k = {5, 25, 50, 75, 100, 125}

over the set of images. One plot per value of h .r

69 - Conclusion

The first comparison considered the

correctness of the three algorithms.

Hybrid algorithm performed slightly better

than the mean shift algorithm, and both

performed significantly better than the

graph-based segmentation.

We can conclude that the mean shift

filtering step is indeed useful, and that the

most promising algorithms are the mean

shift segmentation and the hybrid algorithm.

70 - Conclusion (Cont.)

The second comparison considered stability

with respect to parameters.

The hybrid algorithm showed less variability

when its parameters were changed than the

mean shift segmentation algorithm.

Although the amount of improvement did

decline with increasing values of k, the rate of

decline was very slow.

Although the graph-based segmentation did

show very low variability with k = 5, changing

the value of k decreased its stability drastically.

71 - Conclusion (Cont.)

Finally, we compared the stability of a

particular parameter choice over the set of

images.

Once again we see that the graph-based

algorithm has low variability when k = 5,

however its performance and stability

decrease rapidly with changing values of k.

The comparison between the mean shift

segmentation and the hybrid method is much

closer here, with neither performing

significantly better.

72 - Conclusion (Cont.)

For the three characteristics measured, we

have demonstrated that both the mean

shift segmentation and hybrid

segmentation algorithms can create

realistic segmentations with a wide variety

of parameters.

However the hybrid algorithm has slightly

improved stability.

Thus, we would choose to incorporate the

hybrid method into a larger system.

73