python deep learning - jordi torres.ai
TRANSCRIPT
![Page 1: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/1.jpg)
1
Python Deep LearningIntroducción práctica con Keras y TensorFlow 2
Jordi TORRES.AI
PATC Courses | Barcelona - February 2020
![Page 2: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/2.jpg)
2
https://torres.ai/python-deep-learning/
Transparencias para impartir docencia con el libro #PythonDL
![Page 3: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/3.jpg)
3
Acerca de estas transparencias:
● Versión: 0.8 (Barcelona, 31/01/2020)
○ Borrador actual de las transparencias del libro «Pyhon Deep Learning».
○ Algunas transparencias contienen texto en inglés. Con el tiempo iremos «puliendo» las transparencias (pero creemos que incluso así pueden ser usadas y por ello ya las compartimos)
![Page 4: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/4.jpg)
4
Contenido del curso
PART 1: INTRODUCCIÓN
1. ¿Qué es el Deep Learning?
2. Entorno de trabajo
3. Python y sus librerías
PART 2: FUNDAMENTOS DEL DEEP LEARNING
4. Redes neuronales densamente conectadas
5. Redes neuronales en Keras
6. Cómo se entrena una red neuronal
7. Parámetros e hiperparámetros en redes neuronales
8. Redes neuronales convolucionales
PART 3: TÉCNICAS DEL DEEP LEARNING
9. Etapas de un proyecto Deep Learning
10. Datos para entrenar redes neuronales
11. Data Augmentation y Transfer Learning
12. Arquitecturas avanzadas de redes neuronales
PART 4: DEEP LEARNING GENERATIVO
13. Redes neuronales recurrentes
14. Generative Adversarial Networks
Course contentPART 1: INTRODUCTION
1. What is Deep Learning?
2. Work environment
3. Python and its libraries
PART 2: FUNDAMENTALS OF DEEP LEARNING
4. Densely connected neural networks.
5. Neural networks in Keras
6. How a neural network is trained
7. Parameters and hyperparameters in neural networks
8. Convolutional neural networks.
PART 3: DEEP LEARNING TECHNIQUES
9. Stages of a Deep Learning project
10. Data to train neural networks
11. Data Augmentation and Transfer Learning
12. Advanced neural network architectures
PART 4: GENERATIVE DEEP LEARNING
13. Recurrent neural networks
14. Generative Adversarial Networks
![Page 5: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/5.jpg)
5
Recursos del libro● Página web del libro:
https://JordiTorres.ai/python-deep-learning
● Github del libro:
https://github.com/JordiTorresBCN/python-deep-learning
● Material adicional del libro para descargar:
https://marketing.marcombo.com + código promocional del libro
![Page 6: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/6.jpg)
6
PART 3: TÉCNICAS DEL DEEP LEARNINGPART 3: DEEP LEARNING TECHNIQUES
9. Stages of a Deep Learning project
10. Data to train neural networks
11. Data Augmentation and Transfer Learning
12. Advanced neural network architectures
9. Etapas de un proyecto Deep Learning
10. Datos para entrenar redes neuronales
11. Data Augmentation y Transfer Learning
12. Arquitecturas avanzadas de redes neuronales
![Page 7: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/7.jpg)
7
PART 3: TÉCNICAS DEL DEEP LEARNINGPART 3: DEEP LEARNING TECHNIQUES
9. Stages of a Deep Learning project
10. Data to train neural networks
11. Data Augmentation and Transfer Learning
12. Advanced neural network architectures
9. Etapas de un proyecto Deep Learning
10. Datos para entrenar redes neuronales
11. Data Augmentation y Transfer Learning
12. Arquitecturas avanzadas de redes neuronales
![Page 8: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/8.jpg)
8
• Goal: Builds a model to predict the fuel efficiency of late-1970s and early 1980s automobiles. • To do this, we'll provide the model with a description of many
automobiles from that time period. • This description includes attributes like: cylinders, displacement,
horsepower, and weight.
Case study: fuel efficiency of late-1970s and early1980s automobiles
![Page 9: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/9.jpg)
9
• Dataset source: https://www.kaggle.com/uciml/autompg-dataset• Number of Instances: 398• Attribute Information:
• Miles per Gallon (mpg): continuous• cylinders: multi-valued discrete• displacement: continuous• horsepower: continuous• weight: continuous• acceleration: continuous• model year: multi-valued discrete• origin: multi-valued discrete
Dataset: Auto MPG
![Page 10: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/10.jpg)
10
Preparación
![Page 11: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/11.jpg)
11
Carga de datos
![Page 12: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/12.jpg)
12
![Page 13: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/13.jpg)
13
«Limpieza» de los datos
● Data’s almost never clean!!!
○ So we need to make sure that all our data is good values
○ There are unknown values?
○ DataFrame.isna() : Detect missing values.
○ To keep this initial tutorial simple drop those rows using: DataFrame.dropna()
![Page 14: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/14.jpg)
14
![Page 15: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/15.jpg)
15
Manejo de datos categóricos● La columna de datos Origin no es numérica, sino
categórica, es decir, el 1 significa «USA», el 2 «Europa» y el 3 «Japan»
![Page 16: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/16.jpg)
16
![Page 17: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/17.jpg)
17
Evaluation modelData
• Training set → For training
• Validation set→ For hyperparameter tuning
• Test set → Test model performance
![Page 18: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/18.jpg)
18
80%Training
![Page 19: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/19.jpg)
19
● Separar los valores que queremos predecir
![Page 20: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/20.jpg)
20
Normalizar los datos de entrada● Inspect the data: overall statistics
![Page 21: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/21.jpg)
21
![Page 22: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/22.jpg)
22
Normalizar los datos de entrada
![Page 23: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/23.jpg)
23
![Page 24: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/24.jpg)
24
Definición del modelo
![Page 25: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/25.jpg)
25
Inspeccionar el modelo
![Page 26: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/26.jpg)
26
Configuración del modelo● Función de pérdida (opciones):
![Page 27: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/27.jpg)
27
Loss functions: MAE or MSE ?● Evaluation metrics used for regression differ from
classification
● Both mean squared error (MSE) and mean absolute error (MAE) are used in regression modeling
○ MAE is more robust to outliers since it does not make use of square(because of the square, large errors have relatively greater influenceon MSE than do the smaller error)
○ MSE is more useful if we are concerned about large errors whoseconsequences are much bigger than equivalent smaller ones.
● Up to you! ;-)
![Page 28: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/28.jpg)
28
Configuración del modelo● Optimizador
![Page 29: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/29.jpg)
29
Entrenamiento del modelo
20% data forvalidation
![Page 30: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/30.jpg)
30
● Visualize the model's training progress using the statsstored in the history object
Evaluación del proceso de entrenamiento
![Page 31: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/31.jpg)
31
![Page 32: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/32.jpg)
32
![Page 33: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/33.jpg)
33
Plot of loss on the training and validation datasets over training epochs
![Page 34: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/34.jpg)
34
Overfitting● The gap between the training and validation loss indicates
overfitting● Our graph shows little improvement, or even degradation
in the validation error after about 100 epochs à Overfitting?
![Page 35: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/35.jpg)
35
EarlyStopping
![Page 36: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/36.jpg)
36
![Page 37: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/37.jpg)
37
Evaluación del modelo
![Page 38: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/38.jpg)
38
Evaluación del modelo: ¿Bueno?● error absoluto medio (MAE)
¿un valor más «comprensible» ?
○ 5 millas por galón à 2.26 kilómetros por litro de error.
○ Si lo pasamos al error que representa en litros por 100 km —medida que usamos habitualmente cuando hablamos de consumo de coches—, nos sale un error de unos 0.022 litros.
![Page 39: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/39.jpg)
39
It is time to get your hands dirty!
![Page 40: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/40.jpg)
40
PART 3: TÉCNICAS DEL DEEP LEARNINGPART 3: DEEP LEARNING TECHNIQUES
9. Stages of a Deep Learning project
10. Data to train neural networks
11. Data Augmentation and Transfer Learning
12. Advanced neural network architectures
9. Etapas de un proyecto Deep Learning
10. Datos para entrenar redes neuronales
11. Data Augmentation y Transfer Learning
12. Arquitecturas avanzadas de redes neuronales
![Page 41: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/41.jpg)
41
¿Dónde encontrar datos?● Conjuntos de datos públicos para entrenar redes neuronales
● Conjuntos de datos precargados
● Conjuntos de datos de Kaggle
![Page 42: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/42.jpg)
42
Caso de estudio: «Dogs vs. Cats»
![Page 43: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/43.jpg)
43
(*) Colab carga los datos en el directorio /content
![Page 44: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/44.jpg)
44
!wget --no-check-certificate \https://www.dropbox.com/s/sshnskxxolkrq9h/cats_and_dogs_small.zip?dl=0 \-O /tmp/cats_and_dogs_small.zip
![Page 45: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/45.jpg)
45
Datos para entrenar, validar y probar
![Page 46: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/46.jpg)
46
![Page 47: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/47.jpg)
47
![Page 48: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/48.jpg)
48
Comprobar los datos
![Page 49: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/49.jpg)
49
![Page 50: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/50.jpg)
50
![Page 51: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/51.jpg)
51
Modelo
![Page 52: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/52.jpg)
52
Recordatorio● capas convoluciones operan
sobre tensores 3D, llamados mapas de características (feature maps), con dos ejes espaciales de altura y anchura (height y width), además de un eje de canal (channels)también llamado profundidad (depth).
● Para una imagen de color RGB, la dimensión del eje depth es 3, pues la imagen tiene tres canales: rojo, verde y azul (red, green y blue).
![Page 53: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/53.jpg)
53
![Page 54: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/54.jpg)
54
Configuración del entrenamiento
![Page 55: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/55.jpg)
55
Preprocesado de datos reales con ImageDataGenerator
![Page 56: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/56.jpg)
56
Generadores de datos
![Page 57: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/57.jpg)
57
![Page 58: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/58.jpg)
58
![Page 59: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/59.jpg)
59
![Page 60: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/60.jpg)
60
![Page 61: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/61.jpg)
61
![Page 62: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/62.jpg)
62
Overfitting?
![Page 63: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/63.jpg)
63
Probar el modelo con predict( )
![Page 64: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/64.jpg)
64
Técnicas de prevención del overfitting● Reducir el tamaño del modelo
● Añadir Dropout
● Añadir regularizaciones L1 y/o L2
à Data Augmentationà Transfer Learning
![Page 65: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/65.jpg)
65
It is time to get your hands dirty!
![Page 66: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/66.jpg)
66
PART 3: TÉCNICAS DEL DEEP LEARNINGPART 3: DEEP LEARNING TECHNIQUES
9. Stages of a Deep Learning project
10. Data to train neural networks
11. Data Augmentation and Transfer Learning
12. Advanced neural network architectures
9. Etapas de un proyecto Deep Learning
10. Datos para entrenar redes neuronales
11. Data Augmentation y Transfer Learning
12. Arquitecturas avanzadas de redes neuronales
![Page 67: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/67.jpg)
67
Data Augmentation
![Page 68: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/68.jpg)
68
Configuración de ImageGenerator
![Page 69: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/69.jpg)
69
(*) mismo modelo que antes
![Page 70: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/70.jpg)
70
¡OJO! Reducir durante la clase
![Page 71: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/71.jpg)
71
![Page 72: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/72.jpg)
72
![Page 73: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/73.jpg)
73
(*) ACC previa: 0.739
![Page 74: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/74.jpg)
74
Pre-trained model● Is a saved network that was previously trained on a large
dataset
○ The intuition behind transfer learning is that if this model trained on a large and general enough dataset, this model will effectively serve as a generic model of the visual world. We can leverage these learned featuremaps without having to train a large model on a large dataset by usingthese models as the basis of our own model specific to our task.
● There are 2 scenarios of transfer learning using a pretrainedmodel:
○ Feature Extraction
○ Fine-Tuning
![Page 75: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/75.jpg)
75
Feature Extraction● Use the representations of learned by a previous network
to extract meaningful features from new samples. ● We simply add a new classifier, which will be trained from
scratch, on top of the pretrained model so that we can repurpose the feature maps learned previously for ourdataset.
Weights and biases initialized with trained values
![Page 76: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/76.jpg)
76
Feature Extraction● Do we use the entire pretrained model or just the
convolutional base?
○ We use the feature extraction portion of these pretrainedconvnets (convolutional base) since they are likely to be genericfeatures and learned concepts over a picture.
○ However, the classification part of the pretrained model is oftenspecific to original classification task, and subsequently specific to the set of classes on which the model was trained.
![Page 77: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/77.jpg)
77
Fine-Tuning1. Unfreezing a few of the top layers of a frozen model base
used for feature extraction, 2. and jointly training both the newly added classifier layers
as well as the last layers of the frozen model.
This allows us to "fine tune" the higher order feature representations in addition to our final classifier in order to make them more relevant for the specific task involved.
No Trainable Trainable
![Page 78: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/78.jpg)
78
Feature Extraction
![Page 79: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/79.jpg)
79
![Page 80: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/80.jpg)
80
![Page 81: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/81.jpg)
81
● En Keras congelar una capa: atributo trainable a false
● En Keras los modelos los podemos considerar como capas:
![Page 82: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/82.jpg)
82
![Page 83: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/83.jpg)
83
![Page 84: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/84.jpg)
84
![Page 85: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/85.jpg)
85
![Page 86: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/86.jpg)
86
![Page 87: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/87.jpg)
87
Fine-Tuning1. Unfreezing a few of the top layers of a frozen model base
used for feature extraction, 2. and jointly training both the newly added classifier layers
as well as the last layers of the frozen model.
This allows us to "fine tune" the higher order feature representations in addition to our final classifier in order to make them more relevant for the specific task involved.
No Trainable Trainable
![Page 88: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/88.jpg)
88
Fine-Tuning
![Page 89: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/89.jpg)
89
![Page 90: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/90.jpg)
90
![Page 91: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/91.jpg)
91
![Page 92: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/92.jpg)
92
![Page 93: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/93.jpg)
93
![Page 94: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/94.jpg)
94
![Page 95: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/95.jpg)
95
![Page 96: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/96.jpg)
96
It is time to get your hands dirty!
![Page 97: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/97.jpg)
97
PART 3: TÉCNICAS DEL DEEP LEARNINGPART 3: DEEP LEARNING TECHNIQUES
9. Stages of a Deep Learning project
10. Data to train neural networks
11. Data Augmentation and Transfer Learning
12. Advanced neural network architectures
9. Etapas de un proyecto Deep Learning
10. Datos para entrenar redes neuronales
11. Data Augmentation y Transfer Learning
12. Arquitecturas avanzadas de redes neuronales
![Page 98: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/98.jpg)
98
API funcional de Keras● Mismo ejemplo:
![Page 99: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/99.jpg)
99
![Page 100: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/100.jpg)
100
![Page 101: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/101.jpg)
101
![Page 102: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/102.jpg)
102
![Page 103: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/103.jpg)
103
![Page 104: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/104.jpg)
104
Modelos complejos
![Page 105: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/105.jpg)
105
![Page 106: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/106.jpg)
106
![Page 107: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/107.jpg)
107
Redes neuronales con nombre própio● AlexNet,
● GoogLeNet,
● VGG
● ResNet
● …
![Page 108: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/108.jpg)
108
Acceso a redes preentrenadas● Ejemplo: VGG16
![Page 109: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/109.jpg)
109
![Page 110: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/110.jpg)
110
Uso de redes preentrenadas con Keras● Conjunto de datos CIFAR-10
![Page 111: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/111.jpg)
111
![Page 112: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/112.jpg)
112
ResNet50
Test Acc 0.5725
![Page 113: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/113.jpg)
113
![Page 114: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/114.jpg)
114
![Page 115: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/115.jpg)
115
Resnet50 preentrenada con Imagenet
![Page 116: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/116.jpg)
116
Test Acc 0.7357
![Page 117: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/117.jpg)
117
![Page 118: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/118.jpg)
118
VGG19
![Page 119: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/119.jpg)
119
![Page 120: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/120.jpg)
120
![Page 121: Python Deep Learning - Jordi TORRES.AI](https://reader036.vdocumento.com/reader036/viewer/2022070511/62c1cdc29838834fe616c1ae/html5/thumbnails/121.jpg)
121
It is time to get your hands dirty!