méxico tec de monterrey instituto de inv. eléctricas gustavo arroyo, pablo ibargüengoytia,...
TRANSCRIPT
MéxicoTec de Monterrey
Instituto de Inv. Eléctricas
Gustavo Arroyo,
Pablo Ibargüengoytia,
Eduardo Morales,
L. Enrique Sucar
Reunión Elvira, Albacete 2002
Elvira 2002 L. E. SUCAR 2
• Visión– Endoscopía– Reconocimiento de ademanes
• Aplicaciones industriales– Validación de sensores– Diagnóstico
Elvira 2002 L. E. SUCAR 3
A “general” BN model for Vision
Elvira 2002 L. E. SUCAR 4
Endoscopía
• Endoscopy is a tool for direct observation of the human digestive system
• Recognize “objects” in endoscopy images of the colon for semi-automatic navigation
• Main feature – dark regions
• Main objects – “lumen” & “diverticula”
Elvira 2002 L. E. SUCAR 5
Colon Image
Elvira 2002 L. E. SUCAR 6
Segmentation – dark region
Elvira 2002 L. E. SUCAR 7
RB para endoscopía (parcial)
Elvira 2002 L. E. SUCAR 8
Combinación de conocimiento y datos
• Mejora:– Se parte de una estructura dada por un experto
(subjetiva) y se mejora con datos– Por ejemplo, verificando relaciones de
independencia y alterando la estructura:• Eliminar nodos• Combinar nodos• Insertar nodos
Elvira 2002 L. E. SUCAR 9
Mejora Estructural
YX
Z
X
Z
XY
Z W
Z
YX
Elvira 2002 L. E. SUCAR 10
Semi-automatic Endoscope
Elvira 2002 L. E. SUCAR 11
Endoscopy navegation system
Elvira 2002 L. E. SUCAR 12
Endoscopy navegation system
Elvira 2002 L. E. SUCAR 13
Human activity recognition
• Recognize different human activities based on videos (walk, run, goodbye, attention, etc.)
• Consider the movement of several limbs (arms, legs)
• The movements can differ for different persons or even for the same person
• Several activities can be performed at the same time
• Consider continuos activities
Elvira 2002 L. E. SUCAR 14
Attention
Elvira 2002 L. E. SUCAR 15
Goodbye – Right - Attention
Elvira 2002 L. E. SUCAR 16
Feature extraction
• The color marks (for each limb) are segmented, with its position in each frame
• The directions of movement (discretized in 8 direction) are obtained for each image pair
• A window is used to obtain each sequence of changes (6), which are the observations for the recognition model – a Bayesian network
Elvira 2002 L. E. SUCAR 17
Segmentation
Elvira 2002 L. E. SUCAR 18
Recognition network
Elvira 2002 L. E. SUCAR 19
Gesture recognition
• Recognize 5 dynamic gestures with the right hand
• The gestures are for commanding a mobile robot
• Recognition based on HMM
Elvira 2002 L. E. SUCAR 20
Come
attention
go-right
go-left
stop
Elvira 2002 L. E. SUCAR 21
Feature Extraction
• Skin detection
• Face and hand segmentation
• Hand tracking
• Motion features
Elvira 2002 L. E. SUCAR 22
Segmentation
Radial scan forskin pixel detection
Segmentation by groupingskin pixels in the scan lines
Elvira 2002 L. E. SUCAR 23
TrackingLocate face and hand based on antropometric measures
Track the hand by using the radial scansegmentation in region of interest
Elvira 2002 L. E. SUCAR 24
Features• From each image we obtain the features:
– change in X (X)– change in Y (Y)– change in area (A)– change in size ratio (R)
• Each one is codified in 3 values: (+, 0, -)
X1,Y,1X2,Y2
A1 A2
Elvira 2002 L. E. SUCAR 25
Elvira 2002 L. E. SUCAR 26
St St+1 St+2
DBN for gesture recognition
A
T T+1 T+2
SX,Y A SX,Y A SX,Y
Elvira 2002 L. E. SUCAR 27
Training and Recognition
• The parameters (conditional probabilities) for the DBN are obtained from examples of each gesture using the EM algorithm (similar to Baum-Welch used in HMM)
• For recognition, the posterior probability of each model is obtained by probability propagation (forward)
Elvira 2002 L. E. SUCAR 28
Preliminary Results• Correct recognition:
– come 100 %– attention 66.2 %– stop 68.26 %– go-right 99.25 %– go-left 100 %– average 86%
• Parameter reduction:– HMM: 81 per state– DBN: 15 per state
Elvira 2002 L. E. SUCAR 29
Probabilistic - Logic Networks
• Logic Nodes - logic programs
• Probabilistic Nodes - Bayesian networks
W V
X Y
Z
Z:binary-
relation (X,Y)multi-valued -
relation(X,Y,Z)
Elvira 2002 L. E. SUCAR 30
Inference• Probability of Z depends on values of X and
Y and if R is satisfied:
P(Z) = R(x,y) P(x) P(y)• Reasoning
– off-line: compute the CPT for all values of X and Y (discrete variables with few values) - deterministic node P(Z | X, Y)
– on-line: evaluate during propagation• discrete: compute summation for unknowns • continuos: sampling techniques
Elvira 2002 L. E. SUCAR 31
Gesture Recognition
• Based on relations between the different parts of the arm (hand, elbow, shoulder)
• These relations are expressed as logic nodes in a dynamic logic-probabilisic network
• The model is used for gesture recognition via probability propagation
Elvira 2002 L. E. SUCAR 32
Model
S
Xh Xe
Rhe
Xs
Res
S
Xh
Rhe
Xe Xs
Res
Elvira 2002 L. E. SUCAR 33
Validación de sensores
Elvira 2002 L. E. SUCAR 34
Detection network
dp: position demand fuel valve
pr: real fuel valve position
da: position demand IGVs
pa: real IGV position
ps: gas fuel pressure supply
fg: flow of gas
ga: flow of air
t: temperature
p: pressure
Elvira 2002 L. E. SUCAR 35
Detection Algorithm:
For all nodes:• Instantiate all nodes except one of the nodes
(Ci)• Propagate probabilities and obtain a posterior
probability distribution of Ci
• Read real value of variable represented by Ci
• If P(real value) pvalue then return(ok) else return(faulty)
Elvira 2002 L. E. SUCAR 36
Isolation Network Construction
• Markov blanket (MB): set of variables that makes a variable independent from the others
• EMB(n) = MB(n) + n• A faulty node affects only its EMB• Faults outside the EMB of a node do not affect the
value of the node• The isolation network relates real and apparent
faults: A real fault in a node causes apparent faults in all its EMB
Elvira 2002 L. E. SUCAR 37
Isolation network
Elvira 2002 L. E. SUCAR 38
Isolation Algorithm
• Instantiate the apparent fault node corresponding to Ci in the isolation network
• Propagate probabilities and obtain a posterior probability of all Real fault nodes
Red Temporal
para Diagnóstico de Plantas Eléctricas
Red Temporal
para Diagnóstico de Plantas Eléctricas
Subsistema de una Planta Eléctrica
DRUM
S U P E R H E A T E R S T E A M S Y S T E M
F E E D W A T E R S Y S T E M C O N D E N S E R S Y S T E M
W A T E R - S T E A MG E N E R A T O RS Y S T E M
S T E A M - T U R B I N E S Y S T E M
R E H E A T E RS T E A M S Y S T E M
F E E D A T E R P U M P
F E E D A T E R V A L V E
S P R A Y V A L V E P S T E A M V A L V E
T R U B I N E
S T F
S T T
D R PF
S W F
Elvira 2002 L. E. SUCAR 41
Nodo Temporal
• Nodo que representa un “evento” o cambio de estado de una variable de estado
• Sus valores corresponden a diferentes intervalos de tiempo en que ocurre el cambio
• Ejemplo:– Nodo: incremento de nivel– Valores (3):
• Cambio 0 - 10• Cambio 10 - 50• No cambio
Red bayesiana con nodos temporales
FWF
FWPF LI
SWVF
SWV
SWF
FWVF
FWV FWP STV
STF
DRL
DRP
STT
FWPFOccur 0.58¬Occur 0.42
LIOccur 0.88¬Occur 0.12
FWVFOccur 0.57¬Occur 0.43
SWVFOccur 0.18¬Occur 0.82
FWPtrue, [10-29] = 0.36true, [29-107] = 0.57false, [10-107] = 0.07
STVTrue, [0-18] = 0.69True, [18-29] = 0.20False, [0-29] = 0.11
STFTrue, [52-72] = 0.65True, [72-105] = 0.24False, [52-105] = 0.11
FWVTrue, [28-41] = 0.30True, [41-66] = 0.27False, [28-66] = 0.43
SWVTrue, [20-33] = 0.11True, [33-58] = 0.13False, [20-58] = 0.76
FWFTrue, [25-114] = 0.77True, [114-248] = 0.18False, [25-248] = 0.05
SWFTrue, [108-170] = 0.75True, [170-232] = 0.21False, [108-232] = 0.04
STTDecrement, [10-42] = 0.37Decrement, [42-100] = 0.14Decrement, [100-272] = 0.47False, [10-272] = 0.02
DRPTrue, [30-70] = 0.58True, [70-96] = 0.40False, [30-96] = 0.02
DRLIncrement, [10-27] = 0.49Increment, [27-135] = 0.09Decrement, [22-37] = 0.28Decrement [37-44] = 0.12False, [10-135] = 0.02
Variables
LI=Load increment
FWPF=FW pump failure
FWVF=FW valve failure
SWVF=SW valve failure
STV=Steam valve
FWP=FW pump
FWV=FW valve
SWV=SW valve
STF=Steam flow
FWF=FW flow
SWF=SW flow
DRL=Drum level
DRP=Drum pressure
STT=Steam temperature