このページは http://www.slideshare.net/oulasvir/modelbased-user-interface-optimization-part-i-introduction-at-sicsa-summer-school-on-computational-interaction-2015 の内容を掲載しています。

掲載を希望されないスライド著者の方は、こちらよりご連絡下さい。

1年以上前 (2015/06/24)にアップロードinテクノロジー

Tutorial on Model-Based User Interface Optimization. Part I: INTRODUCTION.

Presented at SICSA Su...

Tutorial on Model-Based User Interface Optimization. Part I: INTRODUCTION.

Presented at SICSA Summer School on Computational Interaction 2015. Note: This one-day lecture is divided into multiple parts.

- Optimization

for User Interface Design

Antti Oulasvirta, Anna Feit

Aalto University

SICSA SUMMER SCHOOL ON COMPUTATIONAL INTERACTION

GLASGOW

JUNE 23, 2015 - BIT.LY/1K9SJ6R
- USERINTERFACES.AALTO.FI
- UI DESIGN AS OPTIMIZATION

DEFINING DESIGN PROBLEMS

ALGORITHMIC SOLUTIONS

ADVANCEDTOPICS

TODAY - SCHEDULE

09:30 Introduction

10:30 Letter Assignment

11:30 Coffee

11:45 Competition!

13:00 Lunch

13:45 Results

14:00 Integer programming

14:30 Solving real problems

15:30 Coffee

15:45 Advanced topics

16:45 Discussion

17:30 Closing - SICSA SUMMER SCHOOL ON COMPUTATIONAL INTERACTION

DAY 2: OPTIMIZATION

INTRODUCTION - MOTIVATION
- design

evaluate

User-Centered Design - Evaluative studies

Design heuristics

User and task analysis

User-Centered Design - A hierarchical menu with 50 items

can be ordered in 10 - Perception

Attention

Memory, learning

Motor control - Designers struggle with

problem-solving [E.g., Nigel Cross] - Costly, slow

Unreliable

Lacks guarantees

Subjective

Non-accumulative

User-centered design - BACKGROUND
- • Mathematics

• Operations research

• Algorithm theory

• Computational complexity theory

• Engineering optimization

• Artiﬁcial intelligence

• Economics and management science

• Software engineering

A MULTI-DISCIPLINARY FIELD

Rao 2009 - CLASSIC PROBLEMS

• Assignment problem

• Closure problem

• Constraint satisfaction

problem

• Cutting stock problem

• Integer programming

• Knapsack problem

• Minimum spanning tree

• Nurse scheduling problem

• Traveling salesman problem

• Vehicle routing problem

• Vehicle rescheduling

problem

• Weapon target assignment

problem

Rao 2009 - ENGINEERING OPTIMIZATION

1. Design of aircraft and aerospace structures for

minimum weight

2. Finding the optimal trajectories of space vehicles

3. Design of civil engineering structures such as

frames, foundations, bridges, towers, chimneys, and

dams for minimum cost

4. Minimum-weight design of structures for

earthquake, wind, and other types of random

loading

5. Design of water resources systems for maximum

beneﬁt

6. Optimal plastic design of structures

7. Optimum design of linkages, cams, gears,

machine tools, and other mechanical components

8. Selection of machining conditions in metal-

cutting processes for minimum production cost

9. Design of material handling equipment, such as

conveyors, trucks, and cranes, for minimum cost

10. Design of pumps, turbines, and heat transfer

equipment for maximum efﬁciency

11. Optimum design of electrical machinery such

as motors, generators, and transformers

12. Optimum design of electrical networks

13. Shortest route taken by a salesperson visiting

various cities during one tour

14. Optimal production planning, controlling, and

scheduling

15.Analysis of statistical data and building empirical

models from experimental

results to obtain the most accurate representation

of the physical phenomenon

16. Optimum design of chemical processing

equipment and plants

17. Design of optimum pipeline networks for

process industries

18. Selection of a site for an industry

19. Planning of maintenance and replacement of

equipment to reduce operating costs

20. Inventory control

21.Allocation of resources or services among

several activities to maximize the beneﬁt

22. Controlling the waiting and idle times and

queueing in production lines to reduce the costs

23. Planning the best strategy to obtain maximum

proﬁt in the presence of a competitor

24. Optimum design of control systems

Rao 2009 - DESIGN OPTIMIZATION

Ford Duratorq engine

• aerospace

• automotive

• chemical

• electronics

• construction

• traffic

• manufacturing

• logistics

• telecommunications - NUMEROUS METHODS

1.2 Historical Development 3

Table 1.1 Methods of Operations Research

Mathematical programming or Stochastic process

optimization techniques techniques Statistical methods

Calculus methods Statistical decision theory Regression analysis

Calculus of variations Markov processes Cluster analysis, pattern

recognitionNonlinear programming Queueing theory

Geometric programming Renewal theory Design of experiments

Quadratic programming Simulation methods Discriminate analysis

(factor analysis)Linear programming Reliability theory

Dynamic programming

Integer programming

Stochastic programming

Separable programming

Multiobjective programming

Network methods: CPM and PERT

Game theory

Modern or nontraditional optimization techniques

Genetic algorithms

Simulated annealing

Ant colony optimization

Particle swarm optimization

Neural networks

Fuzzy optimization

HISTORICAL DEVELOPMENT

The existence of optimization methods can be traced to the days of Newton, Lagrange,

and Cauchy. The development of differential calculus methods of optimization was

Rao 2009 - SCOPE
- USER INTERFACE DESIGN

Graphical user interfaces

Consumer electronics

Automotive interfaces

Input devices

Web user interfaces

Gestural interaction Mobile interfaces

Dialogue interfaces - MODERN METHODS FOR

DESIGN OPTIMIZATION

• Modern methods of engineering optimization

• Genetic algorithms: 1975

• Ant colony optimization:1992

• Particle swarm optimization: 1995

Rao 2009 - COMBINATORIAL OPTIMIZATION

WITH BLACK-BOX METHODS

• High dimensional problems

• Exhaustive search impossible

• Expensive computations

• E.g., user simulations

• Multiple objectives

• Uncertainty, missing knowledge,changing conditions

• Complex or un-analyzed objective functions - GOALS AND APPROACH
- GOALS

Conﬁdence comes with experience

1.Ability to deﬁne UID problems as optimization tasks

2.Ability to apply heuristic methods

3.Knowledge about mathematical approaches

4.Knowledge about state-of-the-art in UI optimization - TEACHING APPROACH

• Breadth plus some depth

• Lectures on methods, applications, and issues

• One longer exercise and several pen-and-paper

exercises - FORMING WORK GROUPS
- BY THIS EVENING, YOU WILL HAVE…

• Knowledge allowing you to replicate basically all

work until 2013

• Including CHI’12 Best paper award and our

UIST’13 paper

• Broader perspectives to methods, issues and

possibilities in UI optimization

• Skills to formulate new problems - YOU WILL BE TORN

• It’s easy!

• Anybody can optimize with black-box methods.

(The question is how well)

• It’s hard!

• Solving real problems require insight in problem

deﬁnition, modeling, and algorithms - PRELIMINARIES

“CRASH COURSE” - WHAT IS OPTIMIZATION?

• The act of obtaining the best result under given circumstances

• The process of ﬁnding the conditions that give the best value

!

• Minimization and maximization are equivalent

Rao 2009 - GENERAL FORM

• The optimum is the design vector X with the lowest

value

• Constraints can be added

• E.g., g(X) < 1

Find X =

8

>><

>>:

x1

x2

· · ·

xn

9

>>=

>>;

which minimizes f(X)

Rao 2009 - COMBINATORIAL PROBLEMS

P = (S, ⌦, f)

Problem

Finite set of

candidate solutions

Constraints

Evaluation function - !

!

Black-box

methods

(e.g., simulated

annealing)

Optimizer

Fitts’ lawBigrams

Objective function

literature. Our approach is directly usable in the available IP

solvers such as CPLEX and Gurobi.

BACKGROUND: THE LETTER ASSIGNMENT PROBLEM

Given n letters and n keyslots, the task of the letter assign-

ment problem is to minimize the average cost ck` of selecting

letter ` after k weighted by the bigram probability pk`:

min

X

k

X

`

pk` · ck` (1)

It is an instance of the Koopmans-Beckman Problem [15, 6],

which is NP-hard. As we will see later, this problem can be

modeled with a quadratic function on binary variables. Thus,

it belongs to a broad class of problems called quadratic as-

signment problem (QAP) where the goal is to minimize the

total pair-wise interaction cost [22, 24].

Except for a few papers using heuristic cost functions (e.g.,

[9]), the Fitts-bigram energy is used (see also [4]). Here, the

Task

n!##

Open#

Save##

Save#as...#

Close#

...#

About#

n!#

A B C D E F G H I J

K L M N O P Q R

S T U V W X Y

A

A B C D E F G H I

J K L M N O P Q R

S T U V W X Y Z

A B C D E F G H I

J K L M N O P Q R

S T U V W X Y Z

Letter assignment

Keyboard optimization

See Karrenbauer & Oulasvirta UIST 2014 - KEY CONCEPTS

• Decision variables

• Constraints (rules)

• Objective function

• types: minimization, maximization, existence,

feasibility

• Names: loss function, cost function, utility function,

ﬁtness function, energy function - OBJECTIVE FUNCTION

• The following operations on the objective function will not

change the optimum solution x*:

• multiplication or division of f(x) by a positive constant c

• addition or subtraction of a positive constant c to/from f(x)

Rao 2009

Figure 1.1 Minimum of f (x) is same as maximum of −f (x).

cf(x)

cf(x)

f(x)

f(x)

f(x)

f(x) f(x)

cf*

f* f*

x*

x

x*

x

c + f(x)

c + f*

Figure 1.2 Optimum solution of cf (x) or c + f (x) same as that of f (x). - DIMENSIONS OF PROBLEMS

• Existence of constraints

• Types of functions involved

• Linear, quadratic and other nonlinear

• Permissible values of design variables

• Binary, integer, continuous, mixed

• Deterministic vs. stochastic problems

• Separability of objective function

• Number of objectives

Rao 2009 - SEARCH CONCEPTS

• Variable assignment

• Constraint satisfaction

• Search neighborhood

• N: S —> 2S

Rao 2009 - SEARCH SPACE

• Size of search space

• Types of solutions

• Feasible solution

• Optimal solution / optimum set

• Local

• Global

• Convergence

Rao 2009 - OPTIMIZATION LANDSCAPE

Global optimum

Local optimum - ORDER NOTATION FOR

COMPUTATIONAL COMPLEXITYIntroduction Algorithms Spanning Trees

Notation for Functions f 2 . . .

!"#$%&"'(')"&%*"+%,+"-'.%"*%*/&0')"&1

!"#$ %&'()*')

!"+&,n$ +&,*-.)/0.%

!"1+&,n2c$ 3&+4+&,*-.)/0.%

&"n$ (56+.'7*-

O"n$ +.'7*-

!"n+&,n$ 85*(.+.'7*-

!"n9$ 85*:-*).%

3&+4'&0.*+

!"cn$ 7;3&'7').*+

!"n<$ =*%)&-.*+ &->%&06.'*)&-.*+

Lecture 1: sheet 26 / 43 Marc Uetz Discrete Optimization

[Marc Uetz - lecture notes on discrete optimization]

Complexity is the number of basic instructions that the algorithm performs until termination - COMPUTATION TIMESSay, we can do 2.2 · 109 operations per second (2.2 GHz)

algorithm A’s computation timea

, for tA(n) =

n log n n log n n2

n3

2n

16 ⇡0 ⇡0 ⇡0 0.002 ms 0.03 ms

64 ⇡0 ⇡0 0.002 ms 0.12 ms 266 y

256 ⇡0 0.001 ms 0.06 ms 7.6 ms 1.6 · 1060

y

4.096 ⇡0 0.02 ms 7.6 ms 31 s ???

65.536 ⇡0 0.47 ms 2 s 78 h ???

16.7 Mio ⇡0 0.2 s 35 h 68066 y ???

a

s=second, ms=1/1000 s, h=hour, y=year

Lecture 1: sheet 33 / 43 Marc Uetz Discrete Optimization

[Marc Uetz - lecture notes on discrete optimization] - EXAMPLE FROM MENU OPTIMIZATION

• Convergence the best method at around 10 minutes

• Although the worse method still improved, catching up would take hours

Figure 5: Temporal performance of the optimizer system

for the Firefox and Seashore case. In our test setup, 10,000

iterations takes about 10 minutes.

Cost-

About 10 minsFrom Bailly et al. UIST’13 - CONSTRAINT SURFACE

3. Bound and acceptable point

4. Bound and unacceptable point

All four types of points are shown in Fig. 1.4.

Figure 1.4 Constraint surfaces in a hypothetical two-dimensional design space. Rao 2009 - PERMUTATION MATRIX

P =

2

4

1 0 0

0 0 1

0 1 0

3

5

Slot 1

Slot 2

Slot 3

Element 1

Element 2

Element 3

P = {1,3,2}

Rao 2009 - APPLICATION

correct optimization algorithms, and how to combine

as the situation may demand.

real world problem

algorithm, model, solution technique

computer implementation

analysis validation,

sensitivity analysis

numerical

methods verification

Rao 2009 - CRITERIA: CHOOSING AN ALGORITHM

1.The type of problem to be solved

2.The availability of a ready-made computer program

3.The calendar time required for the development of a program

4.The necessity of derivatives of the functions f and gj, j = 1,2,...,m

5.The available knowledge about the efﬁciency of the method

6.The accuracy of the solution desired

7.The programming language and quality of coding desired

8.The robustness and dependability in ﬁnding the optimum

9.The generality of the program for solving other problems

10.The ease with which the program can be used

Rao 2009 - SUMMARY

• Basic concepts for single criterion combinatorial

optimization

• Objective function

• Constraints

• Global vs. local optima

• Algorithm a matter of success vs. fail - MODEL-BASED

OPTIMIZATION OF USER

INTERFACES - “The Design Oracle”

Guaranteed quality

Explicit criteria

Perfect psychologist

Fast, inexpensive - Design Space*

Knowledge

* In reality, multidimensional - Results from a study

Design heuristics

Psychophysics functions

Model or simulation - MODEL-BASED INTERFACE

OPTIMIZATION

Designer

Generate

Evaluate

Optimizer

Model

Behavioral sciences

Social sciences

…

Computer science

Design studies - The Psychology of

Human-Computer Interaction

1983

!

Cognitive simulation

!

Raised operations research as a

model discipline

!

“Tortoise of accumulative science

and the hare of intuitive design” - GOMS

Limited applicability

Hard to use

Task deﬁnition

Must be re-run for every design - Formal analysis of

UID problems

A B C D E F G H I J

K L M N O P Q R

S T U V W X Y

A

A B C D E F G H I

J K L M N O P Q R

S T U V W X Y Z

A B C D E F G H I

J K L M N O P Q R

S T U V W X Y Z

Representation

of problem in code

Optimized UI

Combinatorial

optimization Predictive model

of user behavior

Literature,

Experiments - APPLICATION PROCEDURE

1. Choose decision variables

2. Represent the design space in code

3. Formulate the objective function

4. Obtain input values and set weights

5. Choose algorithm

6. Execute

7. Evaluate / test hypothesis - USES OF OPTIMIZATION IN UID

1.Find the best design for a given task

2.Find the best task for a given design

3.Identify how far from the optimum a present UI is

4.Explore design space

5.Change a UI to maximally improve it

6.Find the best design for contradictory conditions - BENEFITS

• Derivation of UI designs from ﬁrst principles

• A problem-solving tool

• Encourages exact formulation of design task

• All assumptions can be scrutinized

• All outcomes are testable “hypotheses” - TYPES OF MODELS

• If-then heuristics (e.g., design heuristics)

• Verbal theories (e.g., mental models theory)

• Conceptual models (e.g., task analysis)

• Statistical models (e.g., technology adoption model)

• Analytical models (e.g., Fitts’ law, models)

• Simulation models (e.g., cognitive models)