このページは http://www.slideshare.net/KentaOono/tensordecomposition の内容を掲載しています。

掲載を希望されないスライド著者の方は、こちらよりご連絡下さい。

- .

Introduction to Tensors

PFI Seminar

Kenta OONO

2012/07/11

Kenta OONO

Introduction to Tensors - Self Introduction

• Kenta OONO (Twitter:@delta2323 )

• PSS(Professional and Support Service) Division

• Jubatus Project

• msgpack-idl, generation of clients

• Survey and Prototyping Team

• 2010 PFI Summer Intern → 2011 Engineer - Agenda

• What is Tensor?

• Decomposition of Matrices and Tensors

• Symmetry Parametrized by Young Diagram - Agenda

• What is Tensor?

• Decomposition of Matrices and Tensors

• Symmetry Parametrized by Young Diagram - Vector as a Special Case of Tensor

Consider a n dimensional vector (Below is the case n = 3)

7

v = 4

(1)

9

When we write this vector as v = (vi ), we have v1 = 7, v2 = 4,

and v3 = 9. As v has One index, this vector is regarded as Rank 1

Tensor.

Note

• We use 1-index notation, not 0-index.

• We somtimes use rank in a diﬀerent meaning, (also in the

context of tensors). - Matrix as a Special Case of Tensor

Consider a n × n Matrix (Below is the case n = 3).

7

10

3

A = 4 −1 −2

(2)

9

4

5

We write this matrix as A = (aij )1≤i,j≤3.

• e.g. a11 = 7, a23 = −2, a32 = 4.

A has Two indices (namely i and j) and these indeices run through

1 to 3. A can be regarded as Rank 2 Tensor.

→ What happens if the number of indices are not just 1 or 2 ? - An Example of Rank 3 Tensor

Consider the set of real numbers X = (xijk )1≤i,j,k≤3. As i, j, k run

throuth 1 to 3 independently, this set contains 3 × 3 × 3 = 27

elements.

If we fix the value of i , we can write this set in a matrix form.

7

10

3

10

2

1

5

−10

18

x1∗∗ = 4 −1 −2 , x2∗∗ = 3

−1 0 , x3∗∗ = −4

1

−2

9

4

5

8

1

4

9

−4 −10

(3)

Note

• In some field, we concatenate these matrix and name it X(1)

• This is called Flattening or Matricing of X . - Is This a Multidimensional Array?

We can realize a tensor as a multiple dimensional array in most

programming languages.

PseudoCode(C like):

int[3][3][3] x;

x[1][1][2] = -1; - Inner Product of Tensors

Let X = (xijk ), Y = (yijk ) be two rank 3 tensors and G = (gij ) be

a symmetric (i.e. g ij = g ji ) and positive definite matrix. We can

introduce an inner product of X and Y by:

n

∑

< X , Y >=

g ai g bj g ck xabc yijk

(4)

a,b,c,i ,j,k=1

Note:

• We can similarly define an inner product of two arbitrary rank

tensor

• X and Y must have same rank. - An Example of Inner Product (Vector)

Let’s consider the case of rank 1 tensor ( = vector). We can write

X = (xi ), Y = (yi ). And we set G = (g ij ) an identity matrix:

1

· · · 0

.

.

G = ..

1

..

(5)

0

· · · 1

In this case, this definition of inner product reduce to ordinal inner

product of two vectors, namely,

∑

∑

< X , Y >=

g ai xayi =

xi yi

(6)

a,i

i - An Example of Tensor

Suppose we have a (smooth) function f : R3 → R. We can derive

tensors of arbitrary rank from this function.

Taking Gradient, we obtain rank 1 tensor. (Please replace (1, 2,

3) with (x , y , z) and vice versa.

∂f

∂x

∇

f = (∇

∂f

i f ) =

(7)

∂y

∂f

∂z

Similarly, Hessian is an example of rank 2 tensor.

∂2f

∂2f

∂2f

∂x∂x

∂x ∂y

∂x ∂z

H(f ) = (H

∂2f

∂2f

∂2f

ij (f )) = ∂y∂x

∂y ∂y

∂y ∂z

(8)

∂2f

∂2f

∂2f

∂z∂x

∂z∂y

∂z∂z - Supersymmetry Property

By taking derivative, we can create an arbitrary rank of tensors.

These tensors are (Super)Symmetric in the sense that

interchange of any two indices remains the tensor identical. For

example,

fxyz = fxzy = fyxz = fyzx = fzxy = fzyx .

(9)

Note:

• Afterward, we write this symmetry as

. - Agenda

• What is Tensor?

• Decomposition of Matrices and Tensors

• Symmetry Parametrized by Young Diagram - Decompose!

It is fundamental in mathematics to decompose some object into

simpler, easier-to-handle objects.

∑

√

• Fourier Expansion : f =

an exp( −1nx)

n∈Z∑

• Legendre Polynomial : P =

ak Pk

k

1

d k

• where Pk(x) =

(x 2 − 1)k

k!2k dx k

• Jordan-H¨older : e = G0 ◁ G1 ◁ G2 ◁ · · · ◁ Gn−1 ◁ Gn = G - Decomposition of Matrix

There are many theories regarding the decomposition (or,

factorization) of matrices.

• SVD

• LU

• QR

• Cholesky

• Jordan

• Diagonalization ...

Although we have theories of factorization of Tensors (e.g. Tucker

Decomposition, higher-order SVD etc.) We do NOT go this

direction. Instead, we consider decomposition of matrix into

Summation of matrix. - Example: Projection to Axis
- Example of Decomposition of Matrix

We can decompose matrix into Symmetric part and

Antisymmetric part.

Example:

7

10

3

7

7

6

0

3

−3

4

−1 −2 =

7 −1 1

+

−3 0 −3

9

4

5

6

1

5

3

3

0

Symmetric Part

Antisymmetric Part

A

Asym

Aanti

(10) - How to Calculate Sym. and Antisym. Part

We can calculate the symmetic and antisymmetric part by simple

calculation (Exercise!).

7

10

3

7

4

9

1

Asym

=

4 −1 −2 + 10 −1 4

2

9

4

5

3

−2 5

(11)

1

asym

=

(a

ij

i j + aji )

2

7

10

3

7

4

9

1

Aanti

=

4 −1 −2 − 10 −1 4

2

9

4

5

3

−2 5

(12)

1

aanti

=

(a

ij

i j − aji )

2 - Picture of Projection to Sym. and Antisym. Part
- Notation Using Young Diagram

We can symbolize symmetry and antisymmetry with Young

Diagram

7

10

3

7

7

6

0

3

−3

4

−1 −2 = 7 −1 1 + −3 0 −3

9

4

5

6

1

5

3

3

0

(13)

A

Asym

Aanti

The number of boxes indicates rank of the tensor (one box for

vector and two boxes for matrix). - Agenda

• What is Tensor?

• Decomposition of Matrices and Tensors

• Symmetry Parametrized by Young Diagram

Note:

• From now on, we concentrate on Rank 3 Tensors (i.e.

k = 3).

• And we assume that n = 3, that is, indices run from 1 to 3. - Symmetrizing Operator

We consider the transformation of tensor:

T

: X = (xijk ) → X ′ = (x′ijk),

(14)

where x ′

= 1 (x

ijk

6

i jk + xi kj + xji k + xjki + xki j + xkji )

(15)

= x(ijk). - Young Diagram and Symmetry of Tensor (Sym. Case)

Let X = (x123) be a tensor of rank 3, we call X Has a Symmetry

of

, if interchange of any of two indices doesn’t change each

entry of X .

Example:

• If X has a symmetry of

, then

x112 = x121 = x211

(16)

x123 = x132 = x213 = x231 = x312 = x321 etc...

• Symmetric matrices have a symmetry of - Property of T

• For all X , T

(X ) has symmetry of

.

• If X has symmetry of

, then T

(X ) = X .

• For all X , T

(T

(X )) = T

(X ). - Antisymmetrizing Operator

Next, we consider Antisymmetrization of tensors:

T

: X = (xijk ) → X ′ = (x′ijk),

(17)

where x ′

= 1 (x

ijk

6

i jk − xikj − xjik + xjki + xkij − xkji )

(18)

= x[ijk]. - Young Diagram and Symmetry of Tensor (Antisym. Case)

Let X = (x123) be a tensor of rank k, we call X has a symmetry of

if interchange of any of two indices change only the sign of

each entry of X .

Example.

• If X has a symmetry of , then

• x123 = −x132 = −x213 = x231 = x312 = −x321

• x112 = 0, because x112 must be equal to −x112.

• Antisymmetric matrix has a symmetry of - Orthogonality of Projection

T

and T

are orthogonal in the sense

T

◦ T (X) = T ◦ T

(X ) = 0.

(19) - Counting Dimension

How many dimension do the vector space consisting of tensors ...

• with symmetry

have ?

• with symmetry

have ? - Dimensions of each subvector spaces.

If n = 3 (that is, i , j and k runs through 1 to 3), then each

dimension is as follows.

But dimension of total space is 3 × 3 × 3 = 27. This is larger than

1 + 10 = 11. - Young Diagram

Young Diagram is an arrangent of boxes of the same size so that:

• two adjacent boxes share one of their side.

• the number of boxes in each row is non-increasing.

• the number of boxes in each column is non-increasing.

Here is an example of Young Diagram with 10 boxes.

(20)

English Notation

French Notation - Decomposition of Tensor (of Rank 3)

We have three types of Young Diagram which have three boxes,

namely,

(21)

,

,

and

Symmetric

Antisymmetric

???

→ What symmetry does

represent? - Coeﬃcients of Symmetrizers

We list coneﬃents of each operator in a tabular form.

Coeﬃcient of xijk

i jk

i kj

ji k

jki

ki j

kji

1/6

1/6

1/6

1/6

1/6

1/6

1/6

-1/6

-1/6

1/6

1/6

-1/6

Sum

1 - Coeﬃcients of Symmetrizers

Coeﬃcient of xijk

i jk

i kj

ji k

jki

ki j

kji

1/6

1/6

1/6

1/6

1/6

1/6

1/6

-1/6

-1/6

1/6

1/6

-1/6

4/6

-2/6

-2/6

Sum

1 - Symmetrizer T

We consider the transformation of tensor:

T

: X = (xijk ) → X ′ = (x′ijk),

(22)

where x ′ = 1 (2x

ijk

3

ijk − xjki − xkij ). - Property of T

• For all X , T

(X ) has symmetry of

.

• If X has symmetry of

, then T

(X ) = X .

(

)

• For all X , T

T

(X )

= T

(X ). - Tensors with Symmetry of

Let X = (xijk ) be a tensor of rank 3. We say X Has a Symmetry

of

, if

1

xijk =

(2xijk − xjki − xkij )

(23)

3

for all 1 ≤ i, j, k ≤ n. - Where This Equation ”Symmetric” ?

Remember that X has a symmetry of

, if

1

xijk =

(2xijk − xjki − xkij )

(24)

3

for all 1 ≤ i, j, k ≤ n.

This equation can be transormed as follows :

1

xijk =

(2xijk − xjki − xkij )

3

⇔ 3x

(25)

ijk = 2xijk − xjki − xkij

⇔ xijk + xjki + xkij = 0 - Projection to Each Symmetrized Tensors
- Decomposition of Higher Rank Tensors

Higher rank tensors are also decompose into symmetric tensors

parametrized by Young Diagram.

⊕

⊕

Rn×n×n×n×n =

⊕

⊕

⊕

⊕

(26) - Summary

• Tensors as a Generalization of Vectors and Matrices.

• Decomposition of Matrices and Tensors

• Symmetry of Tensors Parametrized by Young Diagrams. - References

• T G. Kolda and B W. Bader, Tensor Decompositions and

Applications

• 2009, SIAM Review Vol. 51, No 3, pp 455-500

• D S. Watkins, Fundamentals of Matrix Computations 3rd. ed.

• 2010, A Wiley Series of Texts, Monographs and Tracts

• W. Fulton and J Harris, Representation Theory

• 1991, Graduate Texts in Mathematics Readins in Mathematics.

• And Google ”Representation Theory”, ”Symmetric Group”,

”Young Diagram” and so on.