Deep Learning básico con Keras (Parte 6): NASNet

Publicado por Jesús Utrera Burgal el

KerasNASNet

En este artículo vamos a mostrar la arquitectura NASNet. Ésta fue introducida a finales del año 2017 por el equipo de Google Brain. Puedes consultar el paper en el siguiente enlace https://arxiv.org/abs/1707.07012, pero te aconsejo leer el artículo del blog Google AI.

La arquitectura NASNet se compone de dos tipos de capas: La capa Normal (izquierda), y la capa de Reducción (derecha). Ambas capas fueron diseñadas por AutoML.

Entrenando la arquitectura NASNet

Keras tiene a nuestra disposición esta arquitectura y además, para arquitecturas pequeñas, tiene la versión Mobile que es la que vamos a usar a lo largo de este post.

from keras.applications import nasnet

def create_nasnet():  
  model = nasnet.NASNetMobile(input_shape=(32,32,3), include_top=True, weights=None, input_tensor=None, pooling=None, classes=100)

  return model

Compilamos como hasta ahora...

nasnet_model = create_nasnet()  
nasnet_model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['acc', 'mse'])  

Una vez hecho esto, vamos a ver un resumen del modelo creado.

nasnet_model.summary()  
Layer (type)                    Output Shape         Param #     Connected to  
==================================================================================================
input_1 (InputLayer)            (None, 32, 32, 3)    0  
__________________________________________________________________________________________________  
stem_conv1 (Conv2D)             (None, 15, 15, 32)   864         input_1[0][0]  
__________________________________________________________________________________________________  
stem_bn1 (BatchNormalization)   (None, 15, 15, 32)   128         stem_conv1[0][0]  
__________________________________________________________________________________________________  
activation_1 (Activation)       (None, 15, 15, 32)   0           stem_bn1[0][0]  
__________________________________________________________________________________________________  
reduction_conv_1_stem_1 (Conv2D (None, 15, 15, 11)   352         activation_1[0][0]  
__________________________________________________________________________________________________  
reduction_bn_1_stem_1 (BatchNor (None, 15, 15, 11)   44          reduction_conv_1_stem_1[0][0]  
__________________________________________________________________________________________________  
activation_2 (Activation)       (None, 15, 15, 11)   0           reduction_bn_1_stem_1[0][0]  
__________________________________________________________________________________________________  
activation_4 (Activation)       (None, 15, 15, 32)   0           stem_bn1[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 8, 8, 11)     396         activation_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_1_st (None, 8, 8, 11)     1920        activation_4[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 8, 8, 11)     44          separable_conv_1_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_1 (None, 8, 8, 11)     44          separable_conv_1_reduction_1_stem  
__________________________________________________________________________________________________  
activation_3 (Activation)       (None, 8, 8, 11)     0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
activation_5 (Activation)       (None, 8, 8, 11)     0           separable_conv_1_bn_reduction_1_s  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 8, 8, 11)     396         activation_3[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_1_st (None, 8, 8, 11)     660         activation_5[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 8, 8, 11)     44          separable_conv_2_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_1 (None, 8, 8, 11)     44          separable_conv_2_reduction_1_stem  
__________________________________________________________________________________________________  
activation_6 (Activation)       (None, 15, 15, 32)   0           stem_bn1[0][0]  
__________________________________________________________________________________________________  
reduction_add_1_stem_1 (Add)    (None, 8, 8, 11)     0           separable_conv_2_bn_reduction_lef  
                                                                 separable_conv_2_bn_reduction_1_s
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 8, 8, 11)     1920        activation_6[0][0]  
__________________________________________________________________________________________________  
activation_8 (Activation)       (None, 15, 15, 32)   0           stem_bn1[0][0]  
__________________________________________________________________________________________________  
activation_10 (Activation)      (None, 8, 8, 11)     0           reduction_add_1_stem_1[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 8, 8, 11)     44          separable_conv_1_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 8, 8, 11)     1152        activation_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 8, 8, 11)     220         activation_10[0][0]  
__________________________________________________________________________________________________  
activation_7 (Activation)       (None, 8, 8, 11)     0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 8, 8, 11)     44          separable_conv_1_reduction_right3  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 8, 8, 11)     44          separable_conv_1_reduction_left4_  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 8, 8, 11)     660         activation_7[0][0]  
__________________________________________________________________________________________________  
activation_9 (Activation)       (None, 8, 8, 11)     0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
activation_11 (Activation)      (None, 8, 8, 11)     0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
reduction_left2_stem_1 (MaxPool (None, 8, 8, 11)     0           reduction_bn_1_stem_1[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 8, 8, 11)     44          separable_conv_2_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 8, 8, 11)     396         activation_9[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 8, 8, 11)     220         activation_11[0][0]  
__________________________________________________________________________________________________  
adjust_relu_1_stem_2 (Activatio (None, 15, 15, 32)   0           stem_bn1[0][0]  
__________________________________________________________________________________________________  
reduction_add_2_stem_1 (Add)    (None, 8, 8, 11)     0           reduction_left2_stem_1[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
reduction_left3_stem_1 (Average (None, 8, 8, 11)     0           reduction_bn_1_stem_1[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 8, 8, 11)     44          separable_conv_2_reduction_right3  
__________________________________________________________________________________________________  
reduction_left4_stem_1 (Average (None, 8, 8, 11)     0           reduction_add_1_stem_1[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 8, 8, 11)     44          separable_conv_2_reduction_left4_  
__________________________________________________________________________________________________  
reduction_right5_stem_1 (MaxPoo (None, 8, 8, 11)     0           reduction_bn_1_stem_1[0][0]  
__________________________________________________________________________________________________  
zero_padding2d_1 (ZeroPadding2D (None, 16, 16, 32)   0           adjust_relu_1_stem_2[0][0]  
__________________________________________________________________________________________________  
reduction_add3_stem_1 (Add)     (None, 8, 8, 11)     0           reduction_left3_stem_1[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
add_1 (Add)                     (None, 8, 8, 11)     0           reduction_add_2_stem_1[0][0]  
                                                                 reduction_left4_stem_1[0][0]     
__________________________________________________________________________________________________  
reduction_add4_stem_1 (Add)     (None, 8, 8, 11)     0           separable_conv_2_bn_reduction_lef  
                                                                 reduction_right5_stem_1[0][0]    
__________________________________________________________________________________________________  
cropping2d_1 (Cropping2D)       (None, 15, 15, 32)   0           zero_padding2d_1[0][0]  
__________________________________________________________________________________________________  
reduction_concat_stem_1 (Concat (None, 8, 8, 44)     0           reduction_add_2_stem_1[0][0]  
                                                                 reduction_add3_stem_1[0][0]      
                                                                 add_1[0][0]                      
                                                                 reduction_add4_stem_1[0][0]      
__________________________________________________________________________________________________  
adjust_avg_pool_1_stem_2 (Avera (None, 8, 8, 32)     0           adjust_relu_1_stem_2[0][0]  
__________________________________________________________________________________________________  
adjust_avg_pool_2_stem_2 (Avera (None, 8, 8, 32)     0           cropping2d_1[0][0]  
__________________________________________________________________________________________________  
activation_12 (Activation)      (None, 8, 8, 44)     0           reduction_concat_stem_1[0][0]  
__________________________________________________________________________________________________  
adjust_conv_1_stem_2 (Conv2D)   (None, 8, 8, 11)     352         adjust_avg_pool_1_stem_2[0][0]  
__________________________________________________________________________________________________  
adjust_conv_2_stem_2 (Conv2D)   (None, 8, 8, 11)     352         adjust_avg_pool_2_stem_2[0][0]  
__________________________________________________________________________________________________  
reduction_conv_1_stem_2 (Conv2D (None, 8, 8, 22)     968         activation_12[0][0]  
__________________________________________________________________________________________________  
concatenate_1 (Concatenate)     (None, 8, 8, 22)     0           adjust_conv_1_stem_2[0][0]  
                                                                 adjust_conv_2_stem_2[0][0]       
__________________________________________________________________________________________________  
reduction_bn_1_stem_2 (BatchNor (None, 8, 8, 22)     88          reduction_conv_1_stem_2[0][0]  
__________________________________________________________________________________________________  
adjust_bn_stem_2 (BatchNormaliz (None, 8, 8, 22)     88          concatenate_1[0][0]  
__________________________________________________________________________________________________  
activation_13 (Activation)      (None, 8, 8, 22)     0           reduction_bn_1_stem_2[0][0]  
__________________________________________________________________________________________________  
activation_15 (Activation)      (None, 8, 8, 22)     0           adjust_bn_stem_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 4, 4, 22)     1034        activation_13[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_1_st (None, 4, 4, 22)     1562        activation_15[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 4, 4, 22)     88          separable_conv_1_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_1 (None, 4, 4, 22)     88          separable_conv_1_reduction_1_stem  
__________________________________________________________________________________________________  
activation_14 (Activation)      (None, 4, 4, 22)     0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
activation_16 (Activation)      (None, 4, 4, 22)     0           separable_conv_1_bn_reduction_1_s  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 4, 4, 22)     1034        activation_14[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_1_st (None, 4, 4, 22)     1562        activation_16[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 4, 4, 22)     88          separable_conv_2_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_1 (None, 4, 4, 22)     88          separable_conv_2_reduction_1_stem  
__________________________________________________________________________________________________  
activation_17 (Activation)      (None, 8, 8, 22)     0           adjust_bn_stem_2[0][0]  
__________________________________________________________________________________________________  
reduction_add_1_stem_2 (Add)    (None, 4, 4, 22)     0           separable_conv_2_bn_reduction_lef  
                                                                 separable_conv_2_bn_reduction_1_s
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 4, 4, 22)     1562        activation_17[0][0]  
__________________________________________________________________________________________________  
activation_19 (Activation)      (None, 8, 8, 22)     0           adjust_bn_stem_2[0][0]  
__________________________________________________________________________________________________  
activation_21 (Activation)      (None, 4, 4, 22)     0           reduction_add_1_stem_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 4, 4, 22)     88          separable_conv_1_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 4, 4, 22)     1034        activation_19[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 4, 4, 22)     682         activation_21[0][0]  
__________________________________________________________________________________________________  
activation_18 (Activation)      (None, 4, 4, 22)     0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 4, 4, 22)     88          separable_conv_1_reduction_right3  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 4, 4, 22)     88          separable_conv_1_reduction_left4_  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 4, 4, 22)     1562        activation_18[0][0]  
__________________________________________________________________________________________________  
activation_20 (Activation)      (None, 4, 4, 22)     0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
activation_22 (Activation)      (None, 4, 4, 22)     0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
reduction_left2_stem_2 (MaxPool (None, 4, 4, 22)     0           reduction_bn_1_stem_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 4, 4, 22)     88          separable_conv_2_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 4, 4, 22)     1034        activation_20[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 4, 4, 22)     682         activation_22[0][0]  
__________________________________________________________________________________________________  
adjust_relu_1_0 (Activation)    (None, 8, 8, 44)     0           reduction_concat_stem_1[0][0]  
__________________________________________________________________________________________________  
reduction_add_2_stem_2 (Add)    (None, 4, 4, 22)     0           reduction_left2_stem_2[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
reduction_left3_stem_2 (Average (None, 4, 4, 22)     0           reduction_bn_1_stem_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 4, 4, 22)     88          separable_conv_2_reduction_right3  
__________________________________________________________________________________________________  
reduction_left4_stem_2 (Average (None, 4, 4, 22)     0           reduction_add_1_stem_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 4, 4, 22)     88          separable_conv_2_reduction_left4_  
__________________________________________________________________________________________________  
reduction_right5_stem_2 (MaxPoo (None, 4, 4, 22)     0           reduction_bn_1_stem_2[0][0]  
__________________________________________________________________________________________________  
zero_padding2d_2 (ZeroPadding2D (None, 9, 9, 44)     0           adjust_relu_1_0[0][0]  
__________________________________________________________________________________________________  
reduction_add3_stem_2 (Add)     (None, 4, 4, 22)     0           reduction_left3_stem_2[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
add_2 (Add)                     (None, 4, 4, 22)     0           reduction_add_2_stem_2[0][0]  
                                                                 reduction_left4_stem_2[0][0]     
__________________________________________________________________________________________________  
reduction_add4_stem_2 (Add)     (None, 4, 4, 22)     0           separable_conv_2_bn_reduction_lef  
                                                                 reduction_right5_stem_2[0][0]    
__________________________________________________________________________________________________  
cropping2d_2 (Cropping2D)       (None, 8, 8, 44)     0           zero_padding2d_2[0][0]  
__________________________________________________________________________________________________  
reduction_concat_stem_2 (Concat (None, 4, 4, 88)     0           reduction_add_2_stem_2[0][0]  
                                                                 reduction_add3_stem_2[0][0]      
                                                                 add_2[0][0]                      
                                                                 reduction_add4_stem_2[0][0]      
__________________________________________________________________________________________________  
adjust_avg_pool_1_0 (AveragePoo (None, 4, 4, 44)     0           adjust_relu_1_0[0][0]  
__________________________________________________________________________________________________  
adjust_avg_pool_2_0 (AveragePoo (None, 4, 4, 44)     0           cropping2d_2[0][0]  
__________________________________________________________________________________________________  
adjust_conv_1_0 (Conv2D)        (None, 4, 4, 22)     968         adjust_avg_pool_1_0[0][0]  
__________________________________________________________________________________________________  
adjust_conv_2_0 (Conv2D)        (None, 4, 4, 22)     968         adjust_avg_pool_2_0[0][0]  
__________________________________________________________________________________________________  
activation_23 (Activation)      (None, 4, 4, 88)     0           reduction_concat_stem_2[0][0]  
__________________________________________________________________________________________________  
concatenate_2 (Concatenate)     (None, 4, 4, 44)     0           adjust_conv_1_0[0][0]  
                                                                 adjust_conv_2_0[0][0]            
__________________________________________________________________________________________________  
normal_conv_1_0 (Conv2D)        (None, 4, 4, 44)     3872        activation_23[0][0]  
__________________________________________________________________________________________________  
adjust_bn_0 (BatchNormalization (None, 4, 4, 44)     176         concatenate_2[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_0 (BatchNormalizati (None, 4, 4, 44)     176         normal_conv_1_0[0][0]  
__________________________________________________________________________________________________  
activation_24 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_0[0][0]  
__________________________________________________________________________________________________  
activation_26 (Activation)      (None, 4, 4, 44)     0           adjust_bn_0[0][0]  
__________________________________________________________________________________________________  
activation_28 (Activation)      (None, 4, 4, 44)     0           adjust_bn_0[0][0]  
__________________________________________________________________________________________________  
activation_30 (Activation)      (None, 4, 4, 44)     0           adjust_bn_0[0][0]  
__________________________________________________________________________________________________  
activation_32 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_0[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_0 (None, 4, 4, 44)     3036        activation_24[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 4, 4, 44)     2332        activation_26[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_0 (None, 4, 4, 44)     3036        activation_28[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 4, 4, 44)     2332        activation_30[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_0 (None, 4, 4, 44)     2332        activation_32[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left1_0[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right1_0[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left2_0[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right2_0[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left5_0[0  
__________________________________________________________________________________________________  
activation_25 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_27 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_29 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_31 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_33 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_0 (None, 4, 4, 44)     3036        activation_25[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 4, 4, 44)     2332        activation_27[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_0 (None, 4, 4, 44)     3036        activation_29[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 4, 4, 44)     2332        activation_31[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_0 (None, 4, 4, 44)     2332        activation_33[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left1_0[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right1_0[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left2_0[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right2_0[  
__________________________________________________________________________________________________  
normal_left3_0 (AveragePooling2 (None, 4, 4, 44)     0           normal_bn_1_0[0][0]  
__________________________________________________________________________________________________  
normal_left4_0 (AveragePooling2 (None, 4, 4, 44)     0           adjust_bn_0[0][0]  
__________________________________________________________________________________________________  
normal_right4_0 (AveragePooling (None, 4, 4, 44)     0           adjust_bn_0[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left5_0[0  
__________________________________________________________________________________________________  
normal_add_1_0 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_0 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_0 (Add)            (None, 4, 4, 44)     0           normal_left3_0[0][0]  
                                                                 adjust_bn_0[0][0]                
__________________________________________________________________________________________________  
normal_add_4_0 (Add)            (None, 4, 4, 44)     0           normal_left4_0[0][0]  
                                                                 normal_right4_0[0][0]            
__________________________________________________________________________________________________  
normal_add_5_0 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_0[0][0]              
__________________________________________________________________________________________________  
normal_concat_0 (Concatenate)   (None, 4, 4, 264)    0           adjust_bn_0[0][0]  
                                                                 normal_add_1_0[0][0]             
                                                                 normal_add_2_0[0][0]             
                                                                 normal_add_3_0[0][0]             
                                                                 normal_add_4_0[0][0]             
                                                                 normal_add_5_0[0][0]             
__________________________________________________________________________________________________  
activation_34 (Activation)      (None, 4, 4, 88)     0           reduction_concat_stem_2[0][0]  
__________________________________________________________________________________________________  
activation_35 (Activation)      (None, 4, 4, 264)    0           normal_concat_0[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_1 (Conv2 (None, 4, 4, 44)     3872        activation_34[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_1 (Conv2D)        (None, 4, 4, 44)     11616       activation_35[0][0]  
__________________________________________________________________________________________________  
adjust_bn_1 (BatchNormalization (None, 4, 4, 44)     176         adjust_conv_projection_1[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_1 (BatchNormalizati (None, 4, 4, 44)     176         normal_conv_1_1[0][0]  
__________________________________________________________________________________________________  
activation_36 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_1[0][0]  
__________________________________________________________________________________________________  
activation_38 (Activation)      (None, 4, 4, 44)     0           adjust_bn_1[0][0]  
__________________________________________________________________________________________________  
activation_40 (Activation)      (None, 4, 4, 44)     0           adjust_bn_1[0][0]  
__________________________________________________________________________________________________  
activation_42 (Activation)      (None, 4, 4, 44)     0           adjust_bn_1[0][0]  
__________________________________________________________________________________________________  
activation_44 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_1[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_1 (None, 4, 4, 44)     3036        activation_36[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 4, 4, 44)     2332        activation_38[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_1 (None, 4, 4, 44)     3036        activation_40[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 4, 4, 44)     2332        activation_42[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_1 (None, 4, 4, 44)     2332        activation_44[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left1_1[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right1_1[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left2_1[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right2_1[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left5_1[0  
__________________________________________________________________________________________________  
activation_37 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_39 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_41 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_43 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_45 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_1 (None, 4, 4, 44)     3036        activation_37[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 4, 4, 44)     2332        activation_39[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_1 (None, 4, 4, 44)     3036        activation_41[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 4, 4, 44)     2332        activation_43[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_1 (None, 4, 4, 44)     2332        activation_45[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left1_1[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right1_1[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left2_1[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right2_1[  
__________________________________________________________________________________________________  
normal_left3_1 (AveragePooling2 (None, 4, 4, 44)     0           normal_bn_1_1[0][0]  
__________________________________________________________________________________________________  
normal_left4_1 (AveragePooling2 (None, 4, 4, 44)     0           adjust_bn_1[0][0]  
__________________________________________________________________________________________________  
normal_right4_1 (AveragePooling (None, 4, 4, 44)     0           adjust_bn_1[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left5_1[0  
__________________________________________________________________________________________________  
normal_add_1_1 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_1 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_1 (Add)            (None, 4, 4, 44)     0           normal_left3_1[0][0]  
                                                                 adjust_bn_1[0][0]                
__________________________________________________________________________________________________  
normal_add_4_1 (Add)            (None, 4, 4, 44)     0           normal_left4_1[0][0]  
                                                                 normal_right4_1[0][0]            
__________________________________________________________________________________________________  
normal_add_5_1 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_1[0][0]              
__________________________________________________________________________________________________  
normal_concat_1 (Concatenate)   (None, 4, 4, 264)    0           adjust_bn_1[0][0]  
                                                                 normal_add_1_1[0][0]             
                                                                 normal_add_2_1[0][0]             
                                                                 normal_add_3_1[0][0]             
                                                                 normal_add_4_1[0][0]             
                                                                 normal_add_5_1[0][0]             
__________________________________________________________________________________________________  
activation_46 (Activation)      (None, 4, 4, 264)    0           normal_concat_0[0][0]  
__________________________________________________________________________________________________  
activation_47 (Activation)      (None, 4, 4, 264)    0           normal_concat_1[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_2 (Conv2 (None, 4, 4, 44)     11616       activation_46[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_2 (Conv2D)        (None, 4, 4, 44)     11616       activation_47[0][0]  
__________________________________________________________________________________________________  
adjust_bn_2 (BatchNormalization (None, 4, 4, 44)     176         adjust_conv_projection_2[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_2 (BatchNormalizati (None, 4, 4, 44)     176         normal_conv_1_2[0][0]  
__________________________________________________________________________________________________  
activation_48 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_2[0][0]  
__________________________________________________________________________________________________  
activation_50 (Activation)      (None, 4, 4, 44)     0           adjust_bn_2[0][0]  
__________________________________________________________________________________________________  
activation_52 (Activation)      (None, 4, 4, 44)     0           adjust_bn_2[0][0]  
__________________________________________________________________________________________________  
activation_54 (Activation)      (None, 4, 4, 44)     0           adjust_bn_2[0][0]  
__________________________________________________________________________________________________  
activation_56 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_2 (None, 4, 4, 44)     3036        activation_48[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 4, 4, 44)     2332        activation_50[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_2 (None, 4, 4, 44)     3036        activation_52[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 4, 4, 44)     2332        activation_54[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_2 (None, 4, 4, 44)     2332        activation_56[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left1_2[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right1_2[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left2_2[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right2_2[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left5_2[0  
__________________________________________________________________________________________________  
activation_49 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_51 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_53 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_55 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_57 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_2 (None, 4, 4, 44)     3036        activation_49[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 4, 4, 44)     2332        activation_51[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_2 (None, 4, 4, 44)     3036        activation_53[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 4, 4, 44)     2332        activation_55[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_2 (None, 4, 4, 44)     2332        activation_57[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left1_2[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right1_2[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left2_2[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right2_2[  
__________________________________________________________________________________________________  
normal_left3_2 (AveragePooling2 (None, 4, 4, 44)     0           normal_bn_1_2[0][0]  
__________________________________________________________________________________________________  
normal_left4_2 (AveragePooling2 (None, 4, 4, 44)     0           adjust_bn_2[0][0]  
__________________________________________________________________________________________________  
normal_right4_2 (AveragePooling (None, 4, 4, 44)     0           adjust_bn_2[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left5_2[0  
__________________________________________________________________________________________________  
normal_add_1_2 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_2 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_2 (Add)            (None, 4, 4, 44)     0           normal_left3_2[0][0]  
                                                                 adjust_bn_2[0][0]                
__________________________________________________________________________________________________  
normal_add_4_2 (Add)            (None, 4, 4, 44)     0           normal_left4_2[0][0]  
                                                                 normal_right4_2[0][0]            
__________________________________________________________________________________________________  
normal_add_5_2 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_2[0][0]              
__________________________________________________________________________________________________  
normal_concat_2 (Concatenate)   (None, 4, 4, 264)    0           adjust_bn_2[0][0]  
                                                                 normal_add_1_2[0][0]             
                                                                 normal_add_2_2[0][0]             
                                                                 normal_add_3_2[0][0]             
                                                                 normal_add_4_2[0][0]             
                                                                 normal_add_5_2[0][0]             
__________________________________________________________________________________________________  
activation_58 (Activation)      (None, 4, 4, 264)    0           normal_concat_1[0][0]  
__________________________________________________________________________________________________  
activation_59 (Activation)      (None, 4, 4, 264)    0           normal_concat_2[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_3 (Conv2 (None, 4, 4, 44)     11616       activation_58[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_3 (Conv2D)        (None, 4, 4, 44)     11616       activation_59[0][0]  
__________________________________________________________________________________________________  
adjust_bn_3 (BatchNormalization (None, 4, 4, 44)     176         adjust_conv_projection_3[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_3 (BatchNormalizati (None, 4, 4, 44)     176         normal_conv_1_3[0][0]  
__________________________________________________________________________________________________  
activation_60 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_3[0][0]  
__________________________________________________________________________________________________  
activation_62 (Activation)      (None, 4, 4, 44)     0           adjust_bn_3[0][0]  
__________________________________________________________________________________________________  
activation_64 (Activation)      (None, 4, 4, 44)     0           adjust_bn_3[0][0]  
__________________________________________________________________________________________________  
activation_66 (Activation)      (None, 4, 4, 44)     0           adjust_bn_3[0][0]  
__________________________________________________________________________________________________  
activation_68 (Activation)      (None, 4, 4, 44)     0           normal_bn_1_3[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_3 (None, 4, 4, 44)     3036        activation_60[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 4, 4, 44)     2332        activation_62[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_3 (None, 4, 4, 44)     3036        activation_64[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 4, 4, 44)     2332        activation_66[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_3 (None, 4, 4, 44)     2332        activation_68[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left1_3[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right1_3[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left2_3[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_1_normal_right2_3[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 4, 4, 44)     176         separable_conv_1_normal_left5_3[0  
__________________________________________________________________________________________________  
activation_61 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_63 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_65 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_67 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_69 (Activation)      (None, 4, 4, 44)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_3 (None, 4, 4, 44)     3036        activation_61[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 4, 4, 44)     2332        activation_63[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_3 (None, 4, 4, 44)     3036        activation_65[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 4, 4, 44)     2332        activation_67[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_3 (None, 4, 4, 44)     2332        activation_69[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left1_3[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right1_3[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left2_3[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 4, 4, 44)     176         separable_conv_2_normal_right2_3[  
__________________________________________________________________________________________________  
normal_left3_3 (AveragePooling2 (None, 4, 4, 44)     0           normal_bn_1_3[0][0]  
__________________________________________________________________________________________________  
normal_left4_3 (AveragePooling2 (None, 4, 4, 44)     0           adjust_bn_3[0][0]  
__________________________________________________________________________________________________  
normal_right4_3 (AveragePooling (None, 4, 4, 44)     0           adjust_bn_3[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 4, 4, 44)     176         separable_conv_2_normal_left5_3[0  
__________________________________________________________________________________________________  
normal_add_1_3 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_3 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_3 (Add)            (None, 4, 4, 44)     0           normal_left3_3[0][0]  
                                                                 adjust_bn_3[0][0]                
__________________________________________________________________________________________________  
normal_add_4_3 (Add)            (None, 4, 4, 44)     0           normal_left4_3[0][0]  
                                                                 normal_right4_3[0][0]            
__________________________________________________________________________________________________  
normal_add_5_3 (Add)            (None, 4, 4, 44)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_3[0][0]              
__________________________________________________________________________________________________  
normal_concat_3 (Concatenate)   (None, 4, 4, 264)    0           adjust_bn_3[0][0]  
                                                                 normal_add_1_3[0][0]             
                                                                 normal_add_2_3[0][0]             
                                                                 normal_add_3_3[0][0]             
                                                                 normal_add_4_3[0][0]             
                                                                 normal_add_5_3[0][0]             
__________________________________________________________________________________________________  
activation_71 (Activation)      (None, 4, 4, 264)    0           normal_concat_3[0][0]  
__________________________________________________________________________________________________  
activation_70 (Activation)      (None, 4, 4, 264)    0           normal_concat_2[0][0]  
__________________________________________________________________________________________________  
reduction_conv_1_reduce_4 (Conv (None, 4, 4, 88)     23232       activation_71[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_reduce_4 (None, 4, 4, 88)     23232       activation_70[0][0]  
__________________________________________________________________________________________________  
reduction_bn_1_reduce_4 (BatchN (None, 4, 4, 88)     352         reduction_conv_1_reduce_4[0][0]  
__________________________________________________________________________________________________  
adjust_bn_reduce_4 (BatchNormal (None, 4, 4, 88)     352         adjust_conv_projection_reduce_4[0  
__________________________________________________________________________________________________  
activation_72 (Activation)      (None, 4, 4, 88)     0           reduction_bn_1_reduce_4[0][0]  
__________________________________________________________________________________________________  
activation_74 (Activation)      (None, 4, 4, 88)     0           adjust_bn_reduce_4[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 2, 2, 88)     9944        activation_72[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_1_re (None, 2, 2, 88)     12056       activation_74[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 2, 2, 88)     352         separable_conv_1_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_1 (None, 2, 2, 88)     352         separable_conv_1_reduction_1_redu  
__________________________________________________________________________________________________  
activation_73 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
activation_75 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_reduction_1_r  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 2, 2, 88)     9944        activation_73[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_1_re (None, 2, 2, 88)     12056       activation_75[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 2, 2, 88)     352         separable_conv_2_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_1 (None, 2, 2, 88)     352         separable_conv_2_reduction_1_redu  
__________________________________________________________________________________________________  
activation_76 (Activation)      (None, 4, 4, 88)     0           adjust_bn_reduce_4[0][0]  
__________________________________________________________________________________________________  
reduction_add_1_reduce_4 (Add)  (None, 2, 2, 88)     0           separable_conv_2_bn_reduction_lef  
                                                                 separable_conv_2_bn_reduction_1_r
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 2, 2, 88)     12056       activation_76[0][0]  
__________________________________________________________________________________________________  
activation_78 (Activation)      (None, 4, 4, 88)     0           adjust_bn_reduce_4[0][0]  
__________________________________________________________________________________________________  
activation_80 (Activation)      (None, 2, 2, 88)     0           reduction_add_1_reduce_4[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 2, 2, 88)     352         separable_conv_1_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 2, 2, 88)     9944        activation_78[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 2, 2, 88)     8536        activation_80[0][0]  
__________________________________________________________________________________________________  
activation_77 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 2, 2, 88)     352         separable_conv_1_reduction_right3  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 2, 2, 88)     352         separable_conv_1_reduction_left4_  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 2, 2, 88)     12056       activation_77[0][0]  
__________________________________________________________________________________________________  
activation_79 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
activation_81 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
reduction_left2_reduce_4 (MaxPo (None, 2, 2, 88)     0           reduction_bn_1_reduce_4[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 2, 2, 88)     352         separable_conv_2_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 2, 2, 88)     9944        activation_79[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 2, 2, 88)     8536        activation_81[0][0]  
__________________________________________________________________________________________________  
adjust_relu_1_5 (Activation)    (None, 4, 4, 264)    0           normal_concat_3[0][0]  
__________________________________________________________________________________________________  
reduction_add_2_reduce_4 (Add)  (None, 2, 2, 88)     0           reduction_left2_reduce_4[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
reduction_left3_reduce_4 (Avera (None, 2, 2, 88)     0           reduction_bn_1_reduce_4[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 2, 2, 88)     352         separable_conv_2_reduction_right3  
__________________________________________________________________________________________________  
reduction_left4_reduce_4 (Avera (None, 2, 2, 88)     0           reduction_add_1_reduce_4[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 2, 2, 88)     352         separable_conv_2_reduction_left4_  
__________________________________________________________________________________________________  
reduction_right5_reduce_4 (MaxP (None, 2, 2, 88)     0           reduction_bn_1_reduce_4[0][0]  
__________________________________________________________________________________________________  
zero_padding2d_3 (ZeroPadding2D (None, 5, 5, 264)    0           adjust_relu_1_5[0][0]  
__________________________________________________________________________________________________  
reduction_add3_reduce_4 (Add)   (None, 2, 2, 88)     0           reduction_left3_reduce_4[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
add_3 (Add)                     (None, 2, 2, 88)     0           reduction_add_2_reduce_4[0][0]  
                                                                 reduction_left4_reduce_4[0][0]   
__________________________________________________________________________________________________  
reduction_add4_reduce_4 (Add)   (None, 2, 2, 88)     0           separable_conv_2_bn_reduction_lef  
                                                                 reduction_right5_reduce_4[0][0]  
__________________________________________________________________________________________________  
cropping2d_3 (Cropping2D)       (None, 4, 4, 264)    0           zero_padding2d_3[0][0]  
__________________________________________________________________________________________________  
reduction_concat_reduce_4 (Conc (None, 2, 2, 352)    0           reduction_add_2_reduce_4[0][0]  
                                                                 reduction_add3_reduce_4[0][0]    
                                                                 add_3[0][0]                      
                                                                 reduction_add4_reduce_4[0][0]    
__________________________________________________________________________________________________  
adjust_avg_pool_1_5 (AveragePoo (None, 2, 2, 264)    0           adjust_relu_1_5[0][0]  
__________________________________________________________________________________________________  
adjust_avg_pool_2_5 (AveragePoo (None, 2, 2, 264)    0           cropping2d_3[0][0]  
__________________________________________________________________________________________________  
adjust_conv_1_5 (Conv2D)        (None, 2, 2, 44)     11616       adjust_avg_pool_1_5[0][0]  
__________________________________________________________________________________________________  
adjust_conv_2_5 (Conv2D)        (None, 2, 2, 44)     11616       adjust_avg_pool_2_5[0][0]  
__________________________________________________________________________________________________  
activation_82 (Activation)      (None, 2, 2, 352)    0           reduction_concat_reduce_4[0][0]  
__________________________________________________________________________________________________  
concatenate_3 (Concatenate)     (None, 2, 2, 88)     0           adjust_conv_1_5[0][0]  
                                                                 adjust_conv_2_5[0][0]            
__________________________________________________________________________________________________  
normal_conv_1_5 (Conv2D)        (None, 2, 2, 88)     30976       activation_82[0][0]  
__________________________________________________________________________________________________  
adjust_bn_5 (BatchNormalization (None, 2, 2, 88)     352         concatenate_3[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_5 (BatchNormalizati (None, 2, 2, 88)     352         normal_conv_1_5[0][0]  
__________________________________________________________________________________________________  
activation_83 (Activation)      (None, 2, 2, 88)     0           normal_bn_1_5[0][0]  
__________________________________________________________________________________________________  
activation_85 (Activation)      (None, 2, 2, 88)     0           adjust_bn_5[0][0]  
__________________________________________________________________________________________________  
activation_87 (Activation)      (None, 2, 2, 88)     0           adjust_bn_5[0][0]  
__________________________________________________________________________________________________  
activation_89 (Activation)      (None, 2, 2, 88)     0           adjust_bn_5[0][0]  
__________________________________________________________________________________________________  
activation_91 (Activation)      (None, 2, 2, 88)     0           normal_bn_1_5[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_5 (None, 2, 2, 88)     9944        activation_83[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 2, 2, 88)     8536        activation_85[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_5 (None, 2, 2, 88)     9944        activation_87[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 2, 2, 88)     8536        activation_89[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_5 (None, 2, 2, 88)     8536        activation_91[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left1_5[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right1_5[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left2_5[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right2_5[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left5_5[0  
__________________________________________________________________________________________________  
activation_84 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_86 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_88 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_90 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_92 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_5 (None, 2, 2, 88)     9944        activation_84[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 2, 2, 88)     8536        activation_86[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_5 (None, 2, 2, 88)     9944        activation_88[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 2, 2, 88)     8536        activation_90[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_5 (None, 2, 2, 88)     8536        activation_92[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left1_5[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right1_5[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left2_5[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right2_5[  
__________________________________________________________________________________________________  
normal_left3_5 (AveragePooling2 (None, 2, 2, 88)     0           normal_bn_1_5[0][0]  
__________________________________________________________________________________________________  
normal_left4_5 (AveragePooling2 (None, 2, 2, 88)     0           adjust_bn_5[0][0]  
__________________________________________________________________________________________________  
normal_right4_5 (AveragePooling (None, 2, 2, 88)     0           adjust_bn_5[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left5_5[0  
__________________________________________________________________________________________________  
normal_add_1_5 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_5 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_5 (Add)            (None, 2, 2, 88)     0           normal_left3_5[0][0]  
                                                                 adjust_bn_5[0][0]                
__________________________________________________________________________________________________  
normal_add_4_5 (Add)            (None, 2, 2, 88)     0           normal_left4_5[0][0]  
                                                                 normal_right4_5[0][0]            
__________________________________________________________________________________________________  
normal_add_5_5 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_5[0][0]              
__________________________________________________________________________________________________  
normal_concat_5 (Concatenate)   (None, 2, 2, 528)    0           adjust_bn_5[0][0]  
                                                                 normal_add_1_5[0][0]             
                                                                 normal_add_2_5[0][0]             
                                                                 normal_add_3_5[0][0]             
                                                                 normal_add_4_5[0][0]             
                                                                 normal_add_5_5[0][0]             
__________________________________________________________________________________________________  
activation_93 (Activation)      (None, 2, 2, 352)    0           reduction_concat_reduce_4[0][0]  
__________________________________________________________________________________________________  
activation_94 (Activation)      (None, 2, 2, 528)    0           normal_concat_5[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_6 (Conv2 (None, 2, 2, 88)     30976       activation_93[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_6 (Conv2D)        (None, 2, 2, 88)     46464       activation_94[0][0]  
__________________________________________________________________________________________________  
adjust_bn_6 (BatchNormalization (None, 2, 2, 88)     352         adjust_conv_projection_6[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_6 (BatchNormalizati (None, 2, 2, 88)     352         normal_conv_1_6[0][0]  
__________________________________________________________________________________________________  
activation_95 (Activation)      (None, 2, 2, 88)     0           normal_bn_1_6[0][0]  
__________________________________________________________________________________________________  
activation_97 (Activation)      (None, 2, 2, 88)     0           adjust_bn_6[0][0]  
__________________________________________________________________________________________________  
activation_99 (Activation)      (None, 2, 2, 88)     0           adjust_bn_6[0][0]  
__________________________________________________________________________________________________  
activation_101 (Activation)     (None, 2, 2, 88)     0           adjust_bn_6[0][0]  
__________________________________________________________________________________________________  
activation_103 (Activation)     (None, 2, 2, 88)     0           normal_bn_1_6[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_6 (None, 2, 2, 88)     9944        activation_95[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 2, 2, 88)     8536        activation_97[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_6 (None, 2, 2, 88)     9944        activation_99[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 2, 2, 88)     8536        activation_101[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_6 (None, 2, 2, 88)     8536        activation_103[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left1_6[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right1_6[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left2_6[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right2_6[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left5_6[0  
__________________________________________________________________________________________________  
activation_96 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_98 (Activation)      (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_100 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_102 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_104 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_6 (None, 2, 2, 88)     9944        activation_96[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 2, 2, 88)     8536        activation_98[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_6 (None, 2, 2, 88)     9944        activation_100[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 2, 2, 88)     8536        activation_102[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_6 (None, 2, 2, 88)     8536        activation_104[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left1_6[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right1_6[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left2_6[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right2_6[  
__________________________________________________________________________________________________  
normal_left3_6 (AveragePooling2 (None, 2, 2, 88)     0           normal_bn_1_6[0][0]  
__________________________________________________________________________________________________  
normal_left4_6 (AveragePooling2 (None, 2, 2, 88)     0           adjust_bn_6[0][0]  
__________________________________________________________________________________________________  
normal_right4_6 (AveragePooling (None, 2, 2, 88)     0           adjust_bn_6[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left5_6[0  
__________________________________________________________________________________________________  
normal_add_1_6 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_6 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_6 (Add)            (None, 2, 2, 88)     0           normal_left3_6[0][0]  
                                                                 adjust_bn_6[0][0]                
__________________________________________________________________________________________________  
normal_add_4_6 (Add)            (None, 2, 2, 88)     0           normal_left4_6[0][0]  
                                                                 normal_right4_6[0][0]            
__________________________________________________________________________________________________  
normal_add_5_6 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_6[0][0]              
__________________________________________________________________________________________________  
normal_concat_6 (Concatenate)   (None, 2, 2, 528)    0           adjust_bn_6[0][0]  
                                                                 normal_add_1_6[0][0]             
                                                                 normal_add_2_6[0][0]             
                                                                 normal_add_3_6[0][0]             
                                                                 normal_add_4_6[0][0]             
                                                                 normal_add_5_6[0][0]             
__________________________________________________________________________________________________  
activation_105 (Activation)     (None, 2, 2, 528)    0           normal_concat_5[0][0]  
__________________________________________________________________________________________________  
activation_106 (Activation)     (None, 2, 2, 528)    0           normal_concat_6[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_7 (Conv2 (None, 2, 2, 88)     46464       activation_105[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_7 (Conv2D)        (None, 2, 2, 88)     46464       activation_106[0][0]  
__________________________________________________________________________________________________  
adjust_bn_7 (BatchNormalization (None, 2, 2, 88)     352         adjust_conv_projection_7[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_7 (BatchNormalizati (None, 2, 2, 88)     352         normal_conv_1_7[0][0]  
__________________________________________________________________________________________________  
activation_107 (Activation)     (None, 2, 2, 88)     0           normal_bn_1_7[0][0]  
__________________________________________________________________________________________________  
activation_109 (Activation)     (None, 2, 2, 88)     0           adjust_bn_7[0][0]  
__________________________________________________________________________________________________  
activation_111 (Activation)     (None, 2, 2, 88)     0           adjust_bn_7[0][0]  
__________________________________________________________________________________________________  
activation_113 (Activation)     (None, 2, 2, 88)     0           adjust_bn_7[0][0]  
__________________________________________________________________________________________________  
activation_115 (Activation)     (None, 2, 2, 88)     0           normal_bn_1_7[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_7 (None, 2, 2, 88)     9944        activation_107[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 2, 2, 88)     8536        activation_109[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_7 (None, 2, 2, 88)     9944        activation_111[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 2, 2, 88)     8536        activation_113[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_7 (None, 2, 2, 88)     8536        activation_115[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left1_7[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right1_7[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left2_7[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right2_7[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left5_7[0  
__________________________________________________________________________________________________  
activation_108 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_110 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_112 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_114 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_116 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_7 (None, 2, 2, 88)     9944        activation_108[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 2, 2, 88)     8536        activation_110[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_7 (None, 2, 2, 88)     9944        activation_112[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 2, 2, 88)     8536        activation_114[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_7 (None, 2, 2, 88)     8536        activation_116[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left1_7[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right1_7[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left2_7[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right2_7[  
__________________________________________________________________________________________________  
normal_left3_7 (AveragePooling2 (None, 2, 2, 88)     0           normal_bn_1_7[0][0]  
__________________________________________________________________________________________________  
normal_left4_7 (AveragePooling2 (None, 2, 2, 88)     0           adjust_bn_7[0][0]  
__________________________________________________________________________________________________  
normal_right4_7 (AveragePooling (None, 2, 2, 88)     0           adjust_bn_7[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left5_7[0  
__________________________________________________________________________________________________  
normal_add_1_7 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_7 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_7 (Add)            (None, 2, 2, 88)     0           normal_left3_7[0][0]  
                                                                 adjust_bn_7[0][0]                
__________________________________________________________________________________________________  
normal_add_4_7 (Add)            (None, 2, 2, 88)     0           normal_left4_7[0][0]  
                                                                 normal_right4_7[0][0]            
__________________________________________________________________________________________________  
normal_add_5_7 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_7[0][0]              
__________________________________________________________________________________________________  
normal_concat_7 (Concatenate)   (None, 2, 2, 528)    0           adjust_bn_7[0][0]  
                                                                 normal_add_1_7[0][0]             
                                                                 normal_add_2_7[0][0]             
                                                                 normal_add_3_7[0][0]             
                                                                 normal_add_4_7[0][0]             
                                                                 normal_add_5_7[0][0]             
__________________________________________________________________________________________________  
activation_117 (Activation)     (None, 2, 2, 528)    0           normal_concat_6[0][0]  
__________________________________________________________________________________________________  
activation_118 (Activation)     (None, 2, 2, 528)    0           normal_concat_7[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_8 (Conv2 (None, 2, 2, 88)     46464       activation_117[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_8 (Conv2D)        (None, 2, 2, 88)     46464       activation_118[0][0]  
__________________________________________________________________________________________________  
adjust_bn_8 (BatchNormalization (None, 2, 2, 88)     352         adjust_conv_projection_8[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_8 (BatchNormalizati (None, 2, 2, 88)     352         normal_conv_1_8[0][0]  
__________________________________________________________________________________________________  
activation_119 (Activation)     (None, 2, 2, 88)     0           normal_bn_1_8[0][0]  
__________________________________________________________________________________________________  
activation_121 (Activation)     (None, 2, 2, 88)     0           adjust_bn_8[0][0]  
__________________________________________________________________________________________________  
activation_123 (Activation)     (None, 2, 2, 88)     0           adjust_bn_8[0][0]  
__________________________________________________________________________________________________  
activation_125 (Activation)     (None, 2, 2, 88)     0           adjust_bn_8[0][0]  
__________________________________________________________________________________________________  
activation_127 (Activation)     (None, 2, 2, 88)     0           normal_bn_1_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_8 (None, 2, 2, 88)     9944        activation_119[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 2, 2, 88)     8536        activation_121[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_8 (None, 2, 2, 88)     9944        activation_123[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 2, 2, 88)     8536        activation_125[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_8 (None, 2, 2, 88)     8536        activation_127[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left1_8[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right1_8[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left2_8[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_1_normal_right2_8[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 2, 2, 88)     352         separable_conv_1_normal_left5_8[0  
__________________________________________________________________________________________________  
activation_120 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_122 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_124 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_126 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_128 (Activation)     (None, 2, 2, 88)     0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_8 (None, 2, 2, 88)     9944        activation_120[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 2, 2, 88)     8536        activation_122[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_8 (None, 2, 2, 88)     9944        activation_124[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 2, 2, 88)     8536        activation_126[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_8 (None, 2, 2, 88)     8536        activation_128[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left1_8[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right1_8[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left2_8[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 2, 2, 88)     352         separable_conv_2_normal_right2_8[  
__________________________________________________________________________________________________  
normal_left3_8 (AveragePooling2 (None, 2, 2, 88)     0           normal_bn_1_8[0][0]  
__________________________________________________________________________________________________  
normal_left4_8 (AveragePooling2 (None, 2, 2, 88)     0           adjust_bn_8[0][0]  
__________________________________________________________________________________________________  
normal_right4_8 (AveragePooling (None, 2, 2, 88)     0           adjust_bn_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 2, 2, 88)     352         separable_conv_2_normal_left5_8[0  
__________________________________________________________________________________________________  
normal_add_1_8 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_8 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_8 (Add)            (None, 2, 2, 88)     0           normal_left3_8[0][0]  
                                                                 adjust_bn_8[0][0]                
__________________________________________________________________________________________________  
normal_add_4_8 (Add)            (None, 2, 2, 88)     0           normal_left4_8[0][0]  
                                                                 normal_right4_8[0][0]            
__________________________________________________________________________________________________  
normal_add_5_8 (Add)            (None, 2, 2, 88)     0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_8[0][0]              
__________________________________________________________________________________________________  
normal_concat_8 (Concatenate)   (None, 2, 2, 528)    0           adjust_bn_8[0][0]  
                                                                 normal_add_1_8[0][0]             
                                                                 normal_add_2_8[0][0]             
                                                                 normal_add_3_8[0][0]             
                                                                 normal_add_4_8[0][0]             
                                                                 normal_add_5_8[0][0]             
__________________________________________________________________________________________________  
activation_130 (Activation)     (None, 2, 2, 528)    0           normal_concat_8[0][0]  
__________________________________________________________________________________________________  
activation_129 (Activation)     (None, 2, 2, 528)    0           normal_concat_7[0][0]  
__________________________________________________________________________________________________  
reduction_conv_1_reduce_8 (Conv (None, 2, 2, 176)    92928       activation_130[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_reduce_8 (None, 2, 2, 176)    92928       activation_129[0][0]  
__________________________________________________________________________________________________  
reduction_bn_1_reduce_8 (BatchN (None, 2, 2, 176)    704         reduction_conv_1_reduce_8[0][0]  
__________________________________________________________________________________________________  
adjust_bn_reduce_8 (BatchNormal (None, 2, 2, 176)    704         adjust_conv_projection_reduce_8[0  
__________________________________________________________________________________________________  
activation_131 (Activation)     (None, 2, 2, 176)    0           reduction_bn_1_reduce_8[0][0]  
__________________________________________________________________________________________________  
activation_133 (Activation)     (None, 2, 2, 176)    0           adjust_bn_reduce_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 1, 1, 176)    35376       activation_131[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_1_re (None, 1, 1, 176)    39600       activation_133[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 1, 1, 176)    704         separable_conv_1_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_1 (None, 1, 1, 176)    704         separable_conv_1_reduction_1_redu  
__________________________________________________________________________________________________  
activation_132 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
activation_134 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_reduction_1_r  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 1, 1, 176)    35376       activation_132[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_1_re (None, 1, 1, 176)    39600       activation_134[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 1, 1, 176)    704         separable_conv_2_reduction_left1_  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_1 (None, 1, 1, 176)    704         separable_conv_2_reduction_1_redu  
__________________________________________________________________________________________________  
activation_135 (Activation)     (None, 2, 2, 176)    0           adjust_bn_reduce_8[0][0]  
__________________________________________________________________________________________________  
reduction_add_1_reduce_8 (Add)  (None, 1, 1, 176)    0           separable_conv_2_bn_reduction_lef  
                                                                 separable_conv_2_bn_reduction_1_r
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 1, 1, 176)    39600       activation_135[0][0]  
__________________________________________________________________________________________________  
activation_137 (Activation)     (None, 2, 2, 176)    0           adjust_bn_reduce_8[0][0]  
__________________________________________________________________________________________________  
activation_139 (Activation)     (None, 1, 1, 176)    0           reduction_add_1_reduce_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 1, 1, 176)    704         separable_conv_1_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_1_reduction_righ (None, 1, 1, 176)    35376       activation_137[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_reduction_left (None, 1, 1, 176)    32560       activation_139[0][0]  
__________________________________________________________________________________________________  
activation_136 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_r (None, 1, 1, 176)    704         separable_conv_1_reduction_right3  
__________________________________________________________________________________________________  
separable_conv_1_bn_reduction_l (None, 1, 1, 176)    704         separable_conv_1_reduction_left4_  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 1, 1, 176)    39600       activation_136[0][0]  
__________________________________________________________________________________________________  
activation_138 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_reduction_rig  
__________________________________________________________________________________________________  
activation_140 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_reduction_lef  
__________________________________________________________________________________________________  
reduction_left2_reduce_8 (MaxPo (None, 1, 1, 176)    0           reduction_bn_1_reduce_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 1, 1, 176)    704         separable_conv_2_reduction_right2  
__________________________________________________________________________________________________  
separable_conv_2_reduction_righ (None, 1, 1, 176)    35376       activation_138[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_reduction_left (None, 1, 1, 176)    32560       activation_140[0][0]  
__________________________________________________________________________________________________  
adjust_relu_1_9 (Activation)    (None, 2, 2, 528)    0           normal_concat_8[0][0]  
__________________________________________________________________________________________________  
reduction_add_2_reduce_8 (Add)  (None, 1, 1, 176)    0           reduction_left2_reduce_8[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
reduction_left3_reduce_8 (Avera (None, 1, 1, 176)    0           reduction_bn_1_reduce_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_r (None, 1, 1, 176)    704         separable_conv_2_reduction_right3  
__________________________________________________________________________________________________  
reduction_left4_reduce_8 (Avera (None, 1, 1, 176)    0           reduction_add_1_reduce_8[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_reduction_l (None, 1, 1, 176)    704         separable_conv_2_reduction_left4_  
__________________________________________________________________________________________________  
reduction_right5_reduce_8 (MaxP (None, 1, 1, 176)    0           reduction_bn_1_reduce_8[0][0]  
__________________________________________________________________________________________________  
zero_padding2d_4 (ZeroPadding2D (None, 3, 3, 528)    0           adjust_relu_1_9[0][0]  
__________________________________________________________________________________________________  
reduction_add3_reduce_8 (Add)   (None, 1, 1, 176)    0           reduction_left3_reduce_8[0][0]  
                                                                 separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________  
add_4 (Add)                     (None, 1, 1, 176)    0           reduction_add_2_reduce_8[0][0]  
                                                                 reduction_left4_reduce_8[0][0]   
__________________________________________________________________________________________________  
reduction_add4_reduce_8 (Add)   (None, 1, 1, 176)    0           separable_conv_2_bn_reduction_lef  
                                                                 reduction_right5_reduce_8[0][0]  
__________________________________________________________________________________________________  
cropping2d_4 (Cropping2D)       (None, 2, 2, 528)    0           zero_padding2d_4[0][0]  
__________________________________________________________________________________________________  
reduction_concat_reduce_8 (Conc (None, 1, 1, 704)    0           reduction_add_2_reduce_8[0][0]  
                                                                 reduction_add3_reduce_8[0][0]    
                                                                 add_4[0][0]                      
                                                                 reduction_add4_reduce_8[0][0]    
__________________________________________________________________________________________________  
adjust_avg_pool_1_9 (AveragePoo (None, 1, 1, 528)    0           adjust_relu_1_9[0][0]  
__________________________________________________________________________________________________  
adjust_avg_pool_2_9 (AveragePoo (None, 1, 1, 528)    0           cropping2d_4[0][0]  
__________________________________________________________________________________________________  
adjust_conv_1_9 (Conv2D)        (None, 1, 1, 88)     46464       adjust_avg_pool_1_9[0][0]  
__________________________________________________________________________________________________  
adjust_conv_2_9 (Conv2D)        (None, 1, 1, 88)     46464       adjust_avg_pool_2_9[0][0]  
__________________________________________________________________________________________________  
activation_141 (Activation)     (None, 1, 1, 704)    0           reduction_concat_reduce_8[0][0]  
__________________________________________________________________________________________________  
concatenate_4 (Concatenate)     (None, 1, 1, 176)    0           adjust_conv_1_9[0][0]  
                                                                 adjust_conv_2_9[0][0]            
__________________________________________________________________________________________________  
normal_conv_1_9 (Conv2D)        (None, 1, 1, 176)    123904      activation_141[0][0]  
__________________________________________________________________________________________________  
adjust_bn_9 (BatchNormalization (None, 1, 1, 176)    704         concatenate_4[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_9 (BatchNormalizati (None, 1, 1, 176)    704         normal_conv_1_9[0][0]  
__________________________________________________________________________________________________  
activation_142 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_9[0][0]  
__________________________________________________________________________________________________  
activation_144 (Activation)     (None, 1, 1, 176)    0           adjust_bn_9[0][0]  
__________________________________________________________________________________________________  
activation_146 (Activation)     (None, 1, 1, 176)    0           adjust_bn_9[0][0]  
__________________________________________________________________________________________________  
activation_148 (Activation)     (None, 1, 1, 176)    0           adjust_bn_9[0][0]  
__________________________________________________________________________________________________  
activation_150 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_9[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_9 (None, 1, 1, 176)    35376       activation_142[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 1, 1, 176)    32560       activation_144[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_9 (None, 1, 1, 176)    35376       activation_146[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 1, 1, 176)    32560       activation_148[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_9 (None, 1, 1, 176)    32560       activation_150[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left1_9[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right1_9[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left2_9[0  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right2_9[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left5_9[0  
__________________________________________________________________________________________________  
activation_143 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_145 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_147 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_149 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_151 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_9 (None, 1, 1, 176)    35376       activation_143[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 1, 1, 176)    32560       activation_145[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_9 (None, 1, 1, 176)    35376       activation_147[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 1, 1, 176)    32560       activation_149[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_9 (None, 1, 1, 176)    32560       activation_151[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left1_9[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right1_9[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left2_9[0  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right2_9[  
__________________________________________________________________________________________________  
normal_left3_9 (AveragePooling2 (None, 1, 1, 176)    0           normal_bn_1_9[0][0]  
__________________________________________________________________________________________________  
normal_left4_9 (AveragePooling2 (None, 1, 1, 176)    0           adjust_bn_9[0][0]  
__________________________________________________________________________________________________  
normal_right4_9 (AveragePooling (None, 1, 1, 176)    0           adjust_bn_9[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left5_9[0  
__________________________________________________________________________________________________  
normal_add_1_9 (Add)            (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_9 (Add)            (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_9 (Add)            (None, 1, 1, 176)    0           normal_left3_9[0][0]  
                                                                 adjust_bn_9[0][0]                
__________________________________________________________________________________________________  
normal_add_4_9 (Add)            (None, 1, 1, 176)    0           normal_left4_9[0][0]  
                                                                 normal_right4_9[0][0]            
__________________________________________________________________________________________________  
normal_add_5_9 (Add)            (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_9[0][0]              
__________________________________________________________________________________________________  
normal_concat_9 (Concatenate)   (None, 1, 1, 1056)   0           adjust_bn_9[0][0]  
                                                                 normal_add_1_9[0][0]             
                                                                 normal_add_2_9[0][0]             
                                                                 normal_add_3_9[0][0]             
                                                                 normal_add_4_9[0][0]             
                                                                 normal_add_5_9[0][0]             
__________________________________________________________________________________________________  
activation_152 (Activation)     (None, 1, 1, 704)    0           reduction_concat_reduce_8[0][0]  
__________________________________________________________________________________________________  
activation_153 (Activation)     (None, 1, 1, 1056)   0           normal_concat_9[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_10 (Conv (None, 1, 1, 176)    123904      activation_152[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_10 (Conv2D)       (None, 1, 1, 176)    185856      activation_153[0][0]  
__________________________________________________________________________________________________  
adjust_bn_10 (BatchNormalizatio (None, 1, 1, 176)    704         adjust_conv_projection_10[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_10 (BatchNormalizat (None, 1, 1, 176)    704         normal_conv_1_10[0][0]  
__________________________________________________________________________________________________  
activation_154 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_10[0][0]  
__________________________________________________________________________________________________  
activation_156 (Activation)     (None, 1, 1, 176)    0           adjust_bn_10[0][0]  
__________________________________________________________________________________________________  
activation_158 (Activation)     (None, 1, 1, 176)    0           adjust_bn_10[0][0]  
__________________________________________________________________________________________________  
activation_160 (Activation)     (None, 1, 1, 176)    0           adjust_bn_10[0][0]  
__________________________________________________________________________________________________  
activation_162 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_10[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_1 (None, 1, 1, 176)    35376       activation_154[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 1, 1, 176)    32560       activation_156[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_1 (None, 1, 1, 176)    35376       activation_158[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 1, 1, 176)    32560       activation_160[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_1 (None, 1, 1, 176)    32560       activation_162[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left1_10[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right1_10  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left2_10[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right2_10  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left5_10[  
__________________________________________________________________________________________________  
activation_155 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_157 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_159 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_161 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_163 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_1 (None, 1, 1, 176)    35376       activation_155[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 1, 1, 176)    32560       activation_157[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_1 (None, 1, 1, 176)    35376       activation_159[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 1, 1, 176)    32560       activation_161[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_1 (None, 1, 1, 176)    32560       activation_163[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left1_10[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right1_10  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left2_10[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right2_10  
__________________________________________________________________________________________________  
normal_left3_10 (AveragePooling (None, 1, 1, 176)    0           normal_bn_1_10[0][0]  
__________________________________________________________________________________________________  
normal_left4_10 (AveragePooling (None, 1, 1, 176)    0           adjust_bn_10[0][0]  
__________________________________________________________________________________________________  
normal_right4_10 (AveragePoolin (None, 1, 1, 176)    0           adjust_bn_10[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left5_10[  
__________________________________________________________________________________________________  
normal_add_1_10 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_10 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_10 (Add)           (None, 1, 1, 176)    0           normal_left3_10[0][0]  
                                                                 adjust_bn_10[0][0]               
__________________________________________________________________________________________________  
normal_add_4_10 (Add)           (None, 1, 1, 176)    0           normal_left4_10[0][0]  
                                                                 normal_right4_10[0][0]           
__________________________________________________________________________________________________  
normal_add_5_10 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_10[0][0]             
__________________________________________________________________________________________________  
normal_concat_10 (Concatenate)  (None, 1, 1, 1056)   0           adjust_bn_10[0][0]  
                                                                 normal_add_1_10[0][0]            
                                                                 normal_add_2_10[0][0]            
                                                                 normal_add_3_10[0][0]            
                                                                 normal_add_4_10[0][0]            
                                                                 normal_add_5_10[0][0]            
__________________________________________________________________________________________________  
activation_164 (Activation)     (None, 1, 1, 1056)   0           normal_concat_9[0][0]  
__________________________________________________________________________________________________  
activation_165 (Activation)     (None, 1, 1, 1056)   0           normal_concat_10[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_11 (Conv (None, 1, 1, 176)    185856      activation_164[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_11 (Conv2D)       (None, 1, 1, 176)    185856      activation_165[0][0]  
__________________________________________________________________________________________________  
adjust_bn_11 (BatchNormalizatio (None, 1, 1, 176)    704         adjust_conv_projection_11[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_11 (BatchNormalizat (None, 1, 1, 176)    704         normal_conv_1_11[0][0]  
__________________________________________________________________________________________________  
activation_166 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_11[0][0]  
__________________________________________________________________________________________________  
activation_168 (Activation)     (None, 1, 1, 176)    0           adjust_bn_11[0][0]  
__________________________________________________________________________________________________  
activation_170 (Activation)     (None, 1, 1, 176)    0           adjust_bn_11[0][0]  
__________________________________________________________________________________________________  
activation_172 (Activation)     (None, 1, 1, 176)    0           adjust_bn_11[0][0]  
__________________________________________________________________________________________________  
activation_174 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_11[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_1 (None, 1, 1, 176)    35376       activation_166[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 1, 1, 176)    32560       activation_168[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_1 (None, 1, 1, 176)    35376       activation_170[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 1, 1, 176)    32560       activation_172[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_1 (None, 1, 1, 176)    32560       activation_174[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left1_11[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right1_11  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left2_11[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right2_11  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left5_11[  
__________________________________________________________________________________________________  
activation_167 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_169 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_171 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_173 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_175 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_1 (None, 1, 1, 176)    35376       activation_167[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 1, 1, 176)    32560       activation_169[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_1 (None, 1, 1, 176)    35376       activation_171[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 1, 1, 176)    32560       activation_173[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_1 (None, 1, 1, 176)    32560       activation_175[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left1_11[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right1_11  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left2_11[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right2_11  
__________________________________________________________________________________________________  
normal_left3_11 (AveragePooling (None, 1, 1, 176)    0           normal_bn_1_11[0][0]  
__________________________________________________________________________________________________  
normal_left4_11 (AveragePooling (None, 1, 1, 176)    0           adjust_bn_11[0][0]  
__________________________________________________________________________________________________  
normal_right4_11 (AveragePoolin (None, 1, 1, 176)    0           adjust_bn_11[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left5_11[  
__________________________________________________________________________________________________  
normal_add_1_11 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_11 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_11 (Add)           (None, 1, 1, 176)    0           normal_left3_11[0][0]  
                                                                 adjust_bn_11[0][0]               
__________________________________________________________________________________________________  
normal_add_4_11 (Add)           (None, 1, 1, 176)    0           normal_left4_11[0][0]  
                                                                 normal_right4_11[0][0]           
__________________________________________________________________________________________________  
normal_add_5_11 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_11[0][0]             
__________________________________________________________________________________________________  
normal_concat_11 (Concatenate)  (None, 1, 1, 1056)   0           adjust_bn_11[0][0]  
                                                                 normal_add_1_11[0][0]            
                                                                 normal_add_2_11[0][0]            
                                                                 normal_add_3_11[0][0]            
                                                                 normal_add_4_11[0][0]            
                                                                 normal_add_5_11[0][0]            
__________________________________________________________________________________________________  
activation_176 (Activation)     (None, 1, 1, 1056)   0           normal_concat_10[0][0]  
__________________________________________________________________________________________________  
activation_177 (Activation)     (None, 1, 1, 1056)   0           normal_concat_11[0][0]  
__________________________________________________________________________________________________  
adjust_conv_projection_12 (Conv (None, 1, 1, 176)    185856      activation_176[0][0]  
__________________________________________________________________________________________________  
normal_conv_1_12 (Conv2D)       (None, 1, 1, 176)    185856      activation_177[0][0]  
__________________________________________________________________________________________________  
adjust_bn_12 (BatchNormalizatio (None, 1, 1, 176)    704         adjust_conv_projection_12[0][0]  
__________________________________________________________________________________________________  
normal_bn_1_12 (BatchNormalizat (None, 1, 1, 176)    704         normal_conv_1_12[0][0]  
__________________________________________________________________________________________________  
activation_178 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_12[0][0]  
__________________________________________________________________________________________________  
activation_180 (Activation)     (None, 1, 1, 176)    0           adjust_bn_12[0][0]  
__________________________________________________________________________________________________  
activation_182 (Activation)     (None, 1, 1, 176)    0           adjust_bn_12[0][0]  
__________________________________________________________________________________________________  
activation_184 (Activation)     (None, 1, 1, 176)    0           adjust_bn_12[0][0]  
__________________________________________________________________________________________________  
activation_186 (Activation)     (None, 1, 1, 176)    0           normal_bn_1_12[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left1_1 (None, 1, 1, 176)    35376       activation_178[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right1_ (None, 1, 1, 176)    32560       activation_180[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left2_1 (None, 1, 1, 176)    35376       activation_182[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_right2_ (None, 1, 1, 176)    32560       activation_184[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_normal_left5_1 (None, 1, 1, 176)    32560       activation_186[0][0]  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left1_12[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right1_12  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left2_12[  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_1_normal_right2_12  
__________________________________________________________________________________________________  
separable_conv_1_bn_normal_left (None, 1, 1, 176)    704         separable_conv_1_normal_left5_12[  
__________________________________________________________________________________________________  
activation_179 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left1_  
__________________________________________________________________________________________________  
activation_181 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right1  
__________________________________________________________________________________________________  
activation_183 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left2_  
__________________________________________________________________________________________________  
activation_185 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_right2  
__________________________________________________________________________________________________  
activation_187 (Activation)     (None, 1, 1, 176)    0           separable_conv_1_bn_normal_left5_  
__________________________________________________________________________________________________  
separable_conv_2_normal_left1_1 (None, 1, 1, 176)    35376       activation_179[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right1_ (None, 1, 1, 176)    32560       activation_181[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left2_1 (None, 1, 1, 176)    35376       activation_183[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_right2_ (None, 1, 1, 176)    32560       activation_185[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_normal_left5_1 (None, 1, 1, 176)    32560       activation_187[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left1_12[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right1_12  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left2_12[  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_righ (None, 1, 1, 176)    704         separable_conv_2_normal_right2_12  
__________________________________________________________________________________________________  
normal_left3_12 (AveragePooling (None, 1, 1, 176)    0           normal_bn_1_12[0][0]  
__________________________________________________________________________________________________  
normal_left4_12 (AveragePooling (None, 1, 1, 176)    0           adjust_bn_12[0][0]  
__________________________________________________________________________________________________  
normal_right4_12 (AveragePoolin (None, 1, 1, 176)    0           adjust_bn_12[0][0]  
__________________________________________________________________________________________________  
separable_conv_2_bn_normal_left (None, 1, 1, 176)    704         separable_conv_2_normal_left5_12[  
__________________________________________________________________________________________________  
normal_add_1_12 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left1_  
                                                                 separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________  
normal_add_2_12 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left2_  
                                                                 separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________  
normal_add_3_12 (Add)           (None, 1, 1, 176)    0           normal_left3_12[0][0]  
                                                                 adjust_bn_12[0][0]               
__________________________________________________________________________________________________  
normal_add_4_12 (Add)           (None, 1, 1, 176)    0           normal_left4_12[0][0]  
                                                                 normal_right4_12[0][0]           
__________________________________________________________________________________________________  
normal_add_5_12 (Add)           (None, 1, 1, 176)    0           separable_conv_2_bn_normal_left5_  
                                                                 normal_bn_1_12[0][0]             
__________________________________________________________________________________________________  
normal_concat_12 (Concatenate)  (None, 1, 1, 1056)   0           adjust_bn_12[0][0]  
                                                                 normal_add_1_12[0][0]            
                                                                 normal_add_2_12[0][0]            
                                                                 normal_add_3_12[0][0]            
                                                                 normal_add_4_12[0][0]            
                                                                 normal_add_5_12[0][0]            
__________________________________________________________________________________________________  
activation_188 (Activation)     (None, 1, 1, 1056)   0           normal_concat_12[0][0]  
__________________________________________________________________________________________________  
global_average_pooling2d_1 (Glo (None, 1056)         0           activation_188[0][0]  
__________________________________________________________________________________________________  
predictions (Dense)             (None, 100)          105700      global_average_pooling2d_1[0][0]  
==================================================================================================
Total params: 4,375,416  
Trainable params: 4,338,678  
Non-trainable params: 36,738  
__________________________________________________________________________________________________  

Igual que con la arquitectura DenseNet, tenemos casi 4 millones y medio de parámetros.

Bien, dicho esto, pasamos a entrenar el modelo.

nasnet = nasnet_model.fit(x=x_train, y=y_train, batch_size=32, epochs=10, verbose=1, validation_data=(x_test, y_test), shuffle=True)  
Train on 50000 samples, validate on 10000 samples  
Epoch 1/10  
 50000/50000 [==============================] - 718s 14ms/step - loss: 4.1578 - acc: 0.0711 - mean_squared_error: 0.0098 - val_loss: 3.8363 - val_acc: 0.1054 - val_mean_squared_error: 0.0096
Epoch 2/10  
 50000/50000 [==============================] - 691s 14ms/step - loss: 3.6946 - acc: 0.1287 - mean_squared_error: 0.0095 - val_loss: 3.5996 - val_acc: 0.1448 - val_mean_squared_error: 0.0094
Epoch 3/10  
 50000/50000 [==============================] - 694s 14ms/step - loss: 3.4597 - acc: 0.1673 - mean_squared_error: 0.0093 - val_loss: 3.4186 - val_acc: 0.1760 - val_mean_squared_error: 0.0092
Epoch 4/10  
 50000/50000 [==============================] - 693s 14ms/step - loss: 3.2764 - acc: 0.1962 - mean_squared_error: 0.0091 - val_loss: 3.2867 - val_acc: 0.2003 - val_mean_squared_error: 0.0091
Epoch 5/10  
 50000/50000 [==============================] - 694s 14ms/step - loss: 3.1185 - acc: 0.2273 - mean_squared_error: 0.0089 - val_loss: 3.1641 - val_acc: 0.2273 - val_mean_squared_error: 0.0089
Epoch 6/10  
 50000/50000 [==============================] - 694s 14ms/step - loss: 2.9841 - acc: 0.2518 - mean_squared_error: 0.0087 - val_loss: 3.0661 - val_acc: 0.2469 - val_mean_squared_error: 0.0087
Epoch 7/10  
 50000/50000 [==============================] - 691s 14ms/step - loss: 2.8681 - acc: 0.2726 - mean_squared_error: 0.0085 - val_loss: 3.0061 - val_acc: 0.2569 - val_mean_squared_error: 0.0087
Epoch 8/10  
 50000/50000 [==============================] - 692s 14ms/step - loss: 2.7637 - acc: 0.2955 - mean_squared_error: 0.0083 - val_loss: 2.9539 - val_acc: 0.2651 - val_mean_squared_error: 0.0086
Epoch 9/10  
 50000/50000 [==============================] - 693s 14ms/step - loss: 2.6611 - acc: 0.3161 - mean_squared_error: 0.0081 - val_loss: 2.9056 - val_acc: 0.2813 - val_mean_squared_error: 0.0085
Epoch 10/10  
 50000/50000 [==============================] - 694s 14ms/step - loss: 2.5709 - acc: 0.3337 - mean_squared_error: 0.0080 - val_loss: 2.8702 - val_acc: 0.2827 - val_mean_squared_error: 0.0084

Veamos las métricas obtenidas para el entrenamiento y validación gráficamente.

plt.figure(0)  
plt.plot(nasnet.history['acc'],'r')  
plt.plot(nasnet.history['val_acc'],'g')  
plt.xticks(np.arange(0, 11, 2.0))  
plt.rcParams['figure.figsize'] = (8, 6)  
plt.xlabel("Num of Epochs")  
plt.ylabel("Accuracy")  
plt.title("Training Accuracy vs Validation Accuracy")  
plt.legend(['train','validation'])

plt.figure(1)  
plt.plot(nasnet.history['loss'],'r')  
plt.plot(nasnet.history['val_loss'],'g')  
plt.xticks(np.arange(0, 11, 2.0))  
plt.rcParams['figure.figsize'] = (8, 6)  
plt.xlabel("Num of Epochs")  
plt.ylabel("Loss")  
plt.title("Training Loss vs Validation Loss")  
plt.legend(['train','validation'])

plt.show()  
Accuracy
Loss

El entrenamiento ha dado muy buenos resultados y ha generalizado decentemente. Los resultados a priori parecen tan buenos como los de la DenseNet.

Matriz de confusión

Pasemos ahora a ver la matriz de confusión y las métricas de Accuracy, Recall y F1-score.

Vamos a hacer una predicción sobre el dataset de validación y, a partir de ésta, generamos la matriz de confusión y mostramos las métricas mencionadas anteriormente.

nasnet_pred = nasnet_model.predict(x_test, batch_size=32, verbose=1)  
nasnet_predicted = np.argmax(nasnet_pred, axis=1)

nasnet_cm = confusion_matrix(np.argmax(y_test, axis=1), nasnet_predicted)

# Visualizing of confusion matrix
nasnet_df_cm = pd.DataFrame(nasnet_cm, range(100), range(100))  
plt.figure(figsize = (20,14))  
sn.set(font_scale=1.4) #for label size  
sn.heatmap(nasnet_df_cm, annot=True, annot_kws={"size": 12}) # font size  
plt.show()  
Matriz de confusión

Y por último, mostramos las métricas:

nasnet_report = classification_report(np.argmax(y_test, axis=1), nasnet_predicted)  
print(nasnet_report)

             precision    recall  f1-score   support

          0       0.43      0.60      0.50       100
          1       0.29      0.36      0.32       100
          2       0.21      0.17      0.19       100
          3       0.19      0.18      0.18       100
          4       0.14      0.10      0.12       100
          5       0.28      0.20      0.23       100
          6       0.28      0.25      0.26       100
          7       0.38      0.23      0.29       100
          8       0.28      0.44      0.34       100
          9       0.40      0.30      0.34       100
         10       0.12      0.07      0.09       100
         11       0.41      0.13      0.20       100
         12       0.24      0.22      0.23       100
         13       0.22      0.15      0.18       100
         14       0.14      0.19      0.16       100
         15       0.13      0.11      0.12       100
         16       0.24      0.29      0.26       100
         17       0.41      0.43      0.42       100
         18       0.28      0.23      0.25       100
         19       0.18      0.20      0.19       100
         20       0.35      0.57      0.43       100
         21       0.27      0.52      0.35       100
         22       0.22      0.13      0.16       100
         23       0.46      0.57      0.51       100
         24       0.43      0.50      0.47       100
         25       0.21      0.13      0.16       100
         26       0.27      0.21      0.24       100
         27       0.19      0.21      0.20       100
         28       0.39      0.47      0.43       100
         29       0.28      0.26      0.27       100
         30       0.33      0.41      0.36       100
         31       0.29      0.30      0.29       100
         32       0.22      0.17      0.19       100
         33       0.30      0.31      0.31       100
         34       0.16      0.13      0.15       100
         35       0.10      0.05      0.07       100
         36       0.26      0.27      0.26       100
         37       0.19      0.22      0.21       100
         38       0.11      0.08      0.09       100
         39       0.43      0.43      0.43       100
         40       0.26      0.19      0.22       100
         41       0.50      0.56      0.53       100
         42       0.14      0.32      0.20       100
         43       0.18      0.31      0.23       100
         44       0.17      0.08      0.11       100
         45       0.19      0.13      0.15       100
         46       0.23      0.28      0.25       100
         47       0.48      0.29      0.36       100
         48       0.52      0.52      0.52       100
         49       0.40      0.36      0.38       100
         50       0.07      0.14      0.09       100
         51       0.21      0.13      0.16       100
         52       0.49      0.53      0.51       100
         53       0.41      0.57      0.48       100
         54       0.31      0.46      0.37       100
         55       0.09      0.10      0.10       100
         56       0.35      0.36      0.35       100
         57       0.38      0.26      0.31       100
         58       0.31      0.26      0.28       100
         59       0.23      0.26      0.25       100
         60       0.55      0.62      0.58       100
         61       0.39      0.31      0.35       100
         62       0.41      0.35      0.38       100
         63       0.31      0.22      0.26       100
         64       0.09      0.07      0.08       100
         65       0.12      0.08      0.10       100
         66       0.19      0.12      0.15       100
         67       0.29      0.10      0.15       100
         68       0.79      0.44      0.56       100
         69       0.40      0.55      0.47       100
         70       0.35      0.26      0.30       100
         71       0.51      0.48      0.49       100
         72       0.11      0.06      0.08       100
         73       0.24      0.28      0.26       100
         74       0.12      0.12      0.12       100
         75       0.42      0.56      0.48       100
         76       0.43      0.47      0.45       100
         77       0.16      0.06      0.09       100
         78       0.13      0.17      0.15       100
         79       0.23      0.30      0.26       100
         80       0.18      0.10      0.13       100
         81       0.21      0.29      0.24       100
         82       0.51      0.72      0.60       100
         83       0.24      0.15      0.19       100
         84       0.13      0.19      0.16       100
         85       0.31      0.38      0.34       100
         86       0.31      0.23      0.26       100
         87       0.35      0.39      0.37       100
         88       0.17      0.18      0.18       100
         89       0.35      0.30      0.32       100
         90       0.19      0.23      0.21       100
         91       0.31      0.48      0.37       100
         92       0.21      0.22      0.21       100
         93       0.16      0.16      0.16       100
         94       0.51      0.49      0.50       100
         95       0.38      0.37      0.38       100
         96       0.25      0.33      0.28       100
         97       0.17      0.28      0.21       100
         98       0.17      0.06      0.09       100
         99       0.24      0.20      0.22       100

avg / total       0.28      0.28      0.27     10000  

Curva ROC (tasas de verdaderos positivos y falsos positivos)

Vamos a codificar la curva ROC.

from sklearn.datasets import make_classification  
from sklearn.preprocessing import label_binarize  
from scipy import interp  
from itertools import cycle

n_classes = 100

from sklearn.metrics import roc_curve, auc

# Plot linewidth.
lw = 2

# Compute ROC curve and ROC area for each class
fpr = dict()  
tpr = dict()  
roc_auc = dict()  
for i in range(n_classes):  
    fpr[i], tpr[i], _ = roc_curve(y_test[:, i], nasnet_pred[:, i])
    roc_auc[i] = auc(fpr[i], tpr[i])

# Compute micro-average ROC curve and ROC area
fpr["micro"], tpr["micro"], _ = roc_curve(y_test.ravel(), nasnet_pred.ravel())  
roc_auc["micro"] = auc(fpr["micro"], tpr["micro"])

# Compute macro-average ROC curve and ROC area

# First aggregate all false positive rates
all_fpr = np.unique(np.concatenate([fpr[i] for i in range(n_classes)]))

# Then interpolate all ROC curves at this points
mean_tpr = np.zeros_like(all_fpr)  
for i in range(n_classes):  
    mean_tpr += interp(all_fpr, fpr[i], tpr[i])

# Finally average it and compute AUC
mean_tpr /= n_classes

fpr["macro"] = all_fpr  
tpr["macro"] = mean_tpr  
roc_auc["macro"] = auc(fpr["macro"], tpr["macro"])

# Plot all ROC curves
plt.figure(1)  
plt.plot(fpr["micro"], tpr["micro"],  
         label='micro-average ROC curve (area = {0:0.2f})'
               ''.format(roc_auc["micro"]),
         color='deeppink', linestyle=':', linewidth=4)

plt.plot(fpr["macro"], tpr["macro"],  
         label='macro-average ROC curve (area = {0:0.2f})'
               ''.format(roc_auc["macro"]),
         color='navy', linestyle=':', linewidth=4)

colors = cycle(['aqua', 'darkorange', 'cornflowerblue'])  
for i, color in zip(range(n_classes-97), colors):  
    plt.plot(fpr[i], tpr[i], color=color, lw=lw,
             label='ROC curve of class {0} (area = {1:0.2f})'
             ''.format(i, roc_auc[i]))

plt.plot([0, 1], [0, 1], 'k--', lw=lw)  
plt.xlim([0.0, 1.0])  
plt.ylim([0.0, 1.05])  
plt.xlabel('False Positive Rate')  
plt.ylabel('True Positive Rate')  
plt.title('Some extension of Receiver operating characteristic to multi-class')  
plt.legend(loc="lower right")  
plt.show()


# Zoom in view of the upper left corner.
plt.figure(2)  
plt.xlim(0, 0.2)  
plt.ylim(0.8, 1)  
plt.plot(fpr["micro"], tpr["micro"],  
         label='micro-average ROC curve (area = {0:0.2f})'
               ''.format(roc_auc["micro"]),
         color='deeppink', linestyle=':', linewidth=4)

plt.plot(fpr["macro"], tpr["macro"],  
         label='macro-average ROC curve (area = {0:0.2f})'
               ''.format(roc_auc["macro"]),
         color='navy', linestyle=':', linewidth=4)

colors = cycle(['aqua', 'darkorange', 'cornflowerblue'])  
for i, color in zip(range(3), colors):  
    plt.plot(fpr[i], tpr[i], color=color, lw=lw,
             label='ROC curve of class {0} (area = {1:0.2f})'
             ''.format(i, roc_auc[i]))

plt.plot([0, 1], [0, 1], 'k--', lw=lw)  
plt.xlabel('False Positive Rate')  
plt.ylabel('True Positive Rate')  
plt.title('Some extension of Receiver operating characteristic to multi-class')  
plt.legend(loc="lower right")  
plt.show()  

El resultado para tres clases se muestra en los siguientes gráficos.

Curva ROC para tres clases
Zoom de la curva ROC

Salvaremos los datos del histórico de entrenamiento para compararlos con otros modelos y, además, vamos a salvar el modelo con los pesos entrenados para usarlos en el futuro.

#Modelo
nasnet_model.save(path_base + '/nasnet.h5')

#Histórico
with open(path_base + '/nasnet_history.txt', 'wb') as file_pi:  
  pickle.dump(nasnet.history, file_pi)

A continuación, vamos a comparar las métricas con los modelos anteriores.

plt.figure(0)  
plt.plot(snn_history['val_acc'],'r')  
plt.plot(scnn_history['val_acc'],'g')  
plt.plot(vgg16_history['val_acc'],'b')  
plt.plot(cvgg16_history['val_acc'],'y')  
plt.plot(crn50_history['val_acc'],'gold')  
plt.plot(cdense_history['val_acc'],'m')  
plt.plot(nasnet_history['val_acc'],'c')  
plt.xticks(np.arange(0, 11, 2.0))  
plt.rcParams['figure.figsize'] = (8, 6)  
plt.xlabel("Num of Epochs")  
plt.ylabel("Accuracy")  
plt.title("Models")  
plt.legend(['simple NN','CNN','VGG 16','Custom VGG','Custom ResNet', 'Custom DenseNet', 'Mobile NasNET'])  
Comparativa de modelos - Accuracy
plt.figure(1)  
plt.plot(snn_history['val_loss'],'r')  
plt.plot(scnn_history['val_loss'],'g')  
plt.plot(vgg16_history['val_loss'],'b')  
plt.plot(cvgg16_history['val_loss'],'y')  
plt.plot(crn50_history['val_loss'],'gold')  
plt.plot(cdense_history['val_loss'],'m')  
plt.plot(nasnet_history['val_loss'],'c')  
plt.xticks(np.arange(0, 11, 2.0))  
plt.rcParams['figure.figsize'] = (8, 6)  
plt.xlabel("Num of Epochs")  
plt.ylabel("Loss")  
plt.title("Models")  
plt.legend(['simple NN','CNN','VGG 16','Custom VGG','Custom ResNet', 'Custom DenseNet', 'Mobile NasNET'])  
Comparativa de modelos - Loss
plt.figure(2)  
plt.plot(snn_history['val_mean_squared_error'],'r')  
plt.plot(scnn_history['val_mean_squared_error'],'g')  
plt.plot(vgg16_history['val_mean_squared_error'],'b')  
plt.plot(cvgg16_history['val_mean_squared_error'],'y')  
plt.plot(crn50_history['val_mean_squared_error'],'gold')  
plt.plot(cdense_history['val_mean_squared_error'],'m')  
plt.plot(nasnet_history['val_mean_squared_error'],'c')  
plt.xticks(np.arange(0, 11, 2.0))  
plt.rcParams['figure.figsize'] = (8, 6)  
plt.xlabel("Num of Epochs")  
plt.ylabel("Mean Squared Error")  
plt.title("Models")  
plt.legend(['simple NN','CNN','VGG 16','Custom VGG','Custom ResNet', 'Custom DenseNet', 'Mobile NasNET'])  
Comparativa de modelos - MWSE

Conclusión sobre el experimento

La arquitectura NASNet da unos resultados tan buenos como la anterior DenseNet reduciendo además el número de parámetros a aprender.

¡Síguenos en Twitter para estar al día de nuestros próximos posts!

Autor

Jesús Utrera Burgal

Desarrollador .NET por más de 10 años, en los últimos años me he adentrado en el mundo de Machine Learning, concretamente en el área de Supervised Deep Learning.