Tensorflow模型各部分自定义方式

文章目录

1 自定义损失函数
2 其它自定义函数-正则\激活函数\初始化\Constraint
3 自定义层
4 自定义Metrics
5 关于serialize deserialize get register_keras_serializable的使用

Tensorflow模型的定义有几种不同的方式，Sequentia序列方式，Functional 函数方式及subclassing子类方式，对于像pytorch和paddle的子类方式,如查要把整个模型完整保存需要做一些自定义部分，本文将把各种记录都做个简单的定义
本文参考了tensorflow 和keras官网以及https://github.com/ageron/handson-ml3 下自定义部分的内容。

import numpy as np
import tensorflow as tf 
print(tf.__version__)
gpu = tf.config.list_physical_devices('GPU')[-1]
tf.config.experimental.set_memory_growth(gpu, True)
tf.config.set_visible_devices(gpu,'GPU')
1
2
3
4
5
6

2022-07-26 14:09:09.401799: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.


2.9.1
1
2
3
4

1 自定义损失函数

1.1 第一种方法-函数

def huber_fn(y_true,y_pred):
    error = y_true-y_pred
    is_small_error = tf.abs(error)<1
    squared_error = 0.5*tf.square(error)
    linear_error = tf.abs(error)-0.5
    return tf.where(is_small_error,squared_error,linear_error)
1
2
3
4
5
6

接着预测波斯顿房价

(x_train,y_train),(x_test,y_test) = tf.keras.datasets.boston_housing.load_data()
1

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
x_train_scaled=scaler.fit_transform(x_train)
x_test_scaled= scaler.transform(x_test)
1
2
3
4

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),
    tf.keras.layers.Dense(1)
    ]
)
model.compile(loss=huber_fn,optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6
7
8
9

Epoch 1/10
13/13 [==============================] - 0s 14ms/step - loss: 19.8414 - mae: 20.3408 - val_loss: 20.3195 - val_mae: 20.8195
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 19.4023 - mae: 19.9019 - val_loss: 19.8747 - val_mae: 20.3747
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 18.9530 - mae: 19.4519 - val_loss: 19.4094 - val_mae: 19.9094
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 18.4791 - mae: 18.9776 - val_loss: 18.9208 - val_mae: 19.4203
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 17.9883 - mae: 18.4878 - val_loss: 18.4100 - val_mae: 18.9053
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 17.4719 - mae: 17.9718 - val_loss: 17.8766 - val_mae: 18.3734
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 16.9348 - mae: 17.4325 - val_loss: 17.3165 - val_mae: 17.8153
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 16.3632 - mae: 16.8602 - val_loss: 16.7346 - val_mae: 17.2311
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 15.7597 - mae: 16.2561 - val_loss: 16.1315 - val_mae: 16.6279
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 15.1408 - mae: 15.6358 - val_loss: 15.5066 - val_mae: 15.9913
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

保存模型

model.save('my_model_with_a_custom_loss')
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_loss/assets
1

! ls my_model_with_a_custom_loss
1

assets	keras_metadata.pb  saved_model.pb  variables
1

可以看到模型正常保存,接着进行加载并继续进行训练

newmodel=tf.keras.models.load_model('my_model_with_a_custom_loss')
1

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/tmp/ipykernel_4719/1683424579.py in 
----> 1 newmodel=tf.keras.models.load_model('my_model_with_a_custom_loss')


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    708       if obj is None:
    709         raise ValueError(
--> 710             f'Unknown {printable_module_name}: {object_name}. Please ensure '
    711             'this object is passed to the `custom_objects` argument. See '
    712             'https://www.tensorflow.org/guide/keras/save_and_serialize'


ValueError: Unknown loss function: huber_fn. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

报错信息中可以看到Unknow loss,所以要重新加载模型会报错,按照提示进行操作

newmodel=tf.keras.models.load_model('my_model_with_a_custom_loss',custom_objects={'huber_fn':huber_fn})
1

history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1

Epoch 1/5
13/13 [==============================] - 0s 15ms/step - loss: 14.5342 - mae: 15.0307 - val_loss: 14.9579 - val_mae: 15.4550
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 13.9666 - mae: 14.4617 - val_loss: 14.4416 - val_mae: 14.9335
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 13.4062 - mae: 13.9023 - val_loss: 13.9684 - val_mae: 14.4664
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 12.8451 - mae: 13.3421 - val_loss: 13.5037 - val_mae: 13.9950
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 12.2937 - mae: 12.7887 - val_loss: 13.0708 - val_mae: 13.5672
1
2
3
4
5
6
7
8
9
10

以上就完成的自定义损失函数模型的训练、保存和加载。可以看来损失值是接着原先的值变化的。接着往下看：

1.2 第二种方法-装饰器

def create_huber(threshold=1.0):
    def huber_fn(y_true,y_pred):
        error = y_true-y_pred
        is_small_error = tf.abs(error)<threshold
        squared_error = 0.5*tf.square(error)
        linear_error = threshold*tf.abs(error)-0.5*threshold**2
        return tf.where(is_small_error,squared_error,linear_error)
    return huber_fn
1
2
3
4
5
6
7
8

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),
    tf.keras.layers.Dense(1)
    ]
)
model.compile(loss=create_huber(2.0),optimizer='adam',metrics=["mae"])  #重新compile后，所有状态都会进行重新更新
history2=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6
7
8
9

Epoch 1/10
13/13 [==============================] - 1s 15ms/step - loss: 38.6856 - mae: 20.3408 - val_loss: 39.6391 - val_mae: 20.8196
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 37.8080 - mae: 19.9019 - val_loss: 38.7495 - val_mae: 20.3747
Epoch 3/10
13/13 [==============================] - 0s 6ms/step - loss: 36.9100 - mae: 19.4519 - val_loss: 37.8214 - val_mae: 19.9096
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 35.9649 - mae: 18.9778 - val_loss: 36.8493 - val_mae: 19.4203
Epoch 5/10
13/13 [==============================] - 0s 6ms/step - loss: 34.9810 - mae: 18.4875 - val_loss: 35.8320 - val_mae: 18.9041
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 33.9451 - mae: 17.9707 - val_loss: 34.7692 - val_mae: 18.3724
Epoch 7/10
13/13 [==============================] - 0s 6ms/step - loss: 32.8763 - mae: 17.4321 - val_loss: 33.6509 - val_mae: 17.8151
Epoch 8/10
13/13 [==============================] - 0s 6ms/step - loss: 31.7416 - mae: 16.8602 - val_loss: 32.4894 - val_mae: 17.2308
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 30.5389 - mae: 16.2564 - val_loss: 31.2985 - val_mae: 16.6272
Epoch 10/10
13/13 [==============================] - 0s 7ms/step - loss: 29.3051 - mae: 15.6356 - val_loss: 30.0576 - val_mae: 15.9893
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

model.save('my_model_with_a_custom_loss_threshold_2')
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_loss_threshold_2/assets
1

! ls my_model_with_a_custom_loss_threshold_2
1

assets	keras_metadata.pb  saved_model.pb  variables
1

可以看到，同样是保存成功的

newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_threshold_2/')
1

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/tmp/ipykernel_4719/3766741098.py in 
----> 1 newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_threshold_2/')


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    708       if obj is None:
    709         raise ValueError(
--> 710             f'Unknown {printable_module_name}: {object_name}. Please ensure '
    711             'this object is passed to the `custom_objects` argument. See '
    712             'https://www.tensorflow.org/guide/keras/save_and_serialize'


ValueError: Unknown loss function: huber_fn. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

如果不指定custom objects还是会报错

newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_threshold_2/',custom_objects={'huber_fn':create_huber(2.0)})
1

history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1

Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 28.0896 - mae: 15.0308 - val_loss: 28.9675 - val_mae: 15.4553
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 26.9547 - mae: 14.4610 - val_loss: 27.9318 - val_mae: 14.9332
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 25.8384 - mae: 13.9020 - val_loss: 26.9647 - val_mae: 14.4667
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 24.7175 - mae: 13.3422 - val_loss: 26.0434 - val_mae: 13.9959
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 23.6254 - mae: 12.7892 - val_loss: 25.1706 - val_mae: 13.5669
1
2
3
4
5
6
7
8
9
10

1.3 第三种方法-类

class HuberLoss(tf.keras.losses.Loss):
    def __init__(self,threshold=1.0,**kwargs):
        self.threshold=threshold
        super().__init__(**kwargs)
    def call(self,y_true,y_pred):
        error = y_true-y_pred
        is_small_error = tf.abs(error)<self.threshold
        squared_error = 0.5*tf.square(error)
        linear_error = self.threshold*tf.abs(error)-0.5*self.threshold**2
        return tf.where(is_small_error,squared_error,linear_error)
    def get_config(self):
        base_config = super().get_config()
        all_config = {**base_config,"threshold":self.threshold}
        return all_config
1
2
3
4
5
6
7
8
9
10
11
12
13
14

#再次定义新的模型
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),
    tf.keras.layers.Dense(1)
    ]
)

1
2
3
4
5
6
7
8
9

model.compile(loss=HuberLoss(2.),optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
1
2

Epoch 1/10
13/13 [==============================] - 1s 16ms/step - loss: 38.6856 - mae: 20.3408 - val_loss: 39.6391 - val_mae: 20.8196
Epoch 2/10
13/13 [==============================] - 0s 7ms/step - loss: 37.8080 - mae: 19.9019 - val_loss: 38.7495 - val_mae: 20.3747
Epoch 3/10
13/13 [==============================] - 0s 7ms/step - loss: 36.9100 - mae: 19.4519 - val_loss: 37.8214 - val_mae: 19.9096
Epoch 4/10
13/13 [==============================] - 0s 6ms/step - loss: 35.9649 - mae: 18.9778 - val_loss: 36.8493 - val_mae: 19.4203
Epoch 5/10
13/13 [==============================] - 0s 7ms/step - loss: 34.9810 - mae: 18.4875 - val_loss: 35.8320 - val_mae: 18.9041
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 33.9451 - mae: 17.9707 - val_loss: 34.7692 - val_mae: 18.3724
Epoch 7/10
13/13 [==============================] - 0s 6ms/step - loss: 32.8763 - mae: 17.4321 - val_loss: 33.6509 - val_mae: 17.8151
Epoch 8/10
13/13 [==============================] - 0s 6ms/step - loss: 31.7416 - mae: 16.8602 - val_loss: 32.4894 - val_mae: 17.2308
Epoch 9/10
13/13 [==============================] - 0s 6ms/step - loss: 30.5389 - mae: 16.2564 - val_loss: 31.2985 - val_mae: 16.6272
Epoch 10/10
13/13 [==============================] - 0s 6ms/step - loss: 29.3051 - mae: 15.6356 - val_loss: 30.0576 - val_mae: 15.9893
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

可以看到，和方法二的结果是一样的，tensorflow2.9中加入tf.random.set_seed()等其它优化，使得结果尽可能可以复现

model.save('my_model_with_a_custom_loss_class')
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_loss_class/assets
1

newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_class/')
1

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/tmp/ipykernel_4719/829131420.py in 
----> 1 newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_class/')


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)
    561   if cls is None:
    562     raise ValueError(
--> 563         f'Unknown {printable_module_name}: {class_name}. Please ensure this '
    564         'object is passed to the `custom_objects` argument. See '
    565         'https://www.tensorflow.org/guide/keras/save_and_serialize'


ValueError: Unknown loss function: HuberLoss. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

可以看到，模型还是无法正常加载

newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_class/',custom_objects={"HuberLoss":HuberLoss(2.)})
1

history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))

1
2

Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 28.0885 - mae: 15.0302 - val_loss: 28.9629 - val_mae: 15.4528
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 26.9527 - mae: 14.4601 - val_loss: 27.9266 - val_mae: 14.9309
Epoch 3/5
13/13 [==============================] - 0s 6ms/step - loss: 25.8358 - mae: 13.9006 - val_loss: 26.9588 - val_mae: 14.4639
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 24.7142 - mae: 13.3406 - val_loss: 26.0373 - val_mae: 13.9925
Epoch 5/5
13/13 [==============================] - 0s 7ms/step - loss: 23.6216 - mae: 12.7874 - val_loss: 25.1644 - val_mae: 13.5636
1
2
3
4
5
6
7
8
9
10

1.4 总结

以上三种方法，效果是相同的,但都有一个不足,那就是在恢复模型时必须要有自定义部分的原始代码,这给模型部署带来不便,要是想要部署一个模型还要带上这个自定义函数,这是很不合适的后边会写处理方式.

2 其它自定义函数-正则\激活函数\初始化\Constraint

2.1 函数定义-正则\激活函数\初始化\Constraint

#激活函数
def my_softplus(z):
    return tf.math.log(1.0+tf.exp(z))
#初始化函数
def my_glorot_initializer(shape,dtype=tf.float32):
    stddev = tf.sqrt(2./(shape[0]+shape[1]))
    return tf.random.normal(shape,stddev=stddev,dtype=dtype)
#正则化
def my_l1_regularizer(weights):
    return tf.reduce_sum(tf.abs(0.01*weights))
#constraint
def my_positive_weights(weights):
    return tf.where(weights<0.,tf.zeros_like(weights),weights)
1
2
3
4
5
6
7
8
9
10
11
12
13

layer = tf.keras.layers.Dense(1,activation=my_softplus,
                            kernel_initializer=my_glorot_initializer,
                            kernel_regularizer=my_l1_regularizer,
                            kernel_constraint=my_positive_weights
                            )
1
2
3
4
5

layer.get_config()
1

{'name': 'dense_6',
 'trainable': True,
 'dtype': 'float32',
 'units': 1,
 'activation': 'my_softplus',
 'use_bias': True,
 'kernel_initializer': 'my_glorot_initializer',
 'bias_initializer': {'class_name': 'Zeros', 'config': {}},
 'kernel_regularizer': 'my_l1_regularizer',
 'bias_regularizer': None,
 'activity_regularizer': None,
 'kernel_constraint': 'my_positive_weights',
 'bias_constraint': None}
1
2
3
4
5
6
7
8
9
10
11
12
13

接着定义模型,训练,保存,加载,再训练

tf.random.set_seed(42)
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),
    tf.keras.layers.Dense(1,activation=my_softplus,
                            kernel_initializer=my_glorot_initializer,
                            kernel_regularizer=my_l1_regularizer,
                            kernel_constraint=my_positive_weights
                            )
])
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
model.save('my_model_with_a_custom_parts')
1
2
3
4
5
6
7
8
9
10
11
12

Epoch 1/10
13/13 [==============================] - 1s 14ms/step - loss: 515.5219 - mae: 20.7559 - val_loss: 531.4690 - val_mae: 21.1663
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 501.5470 - mae: 20.4198 - val_loss: 519.0331 - val_mae: 20.8716
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 488.6118 - mae: 20.1095 - val_loss: 504.7845 - val_mae: 20.5305
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 473.7936 - mae: 19.7504 - val_loss: 488.5923 - val_mae: 20.1391
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 456.8321 - mae: 19.3409 - val_loss: 470.3868 - val_mae: 19.6946
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 437.9981 - mae: 18.8721 - val_loss: 450.0984 - val_mae: 19.1896
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 417.4472 - mae: 18.3609 - val_loss: 428.3565 - val_mae: 18.6335
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 395.6647 - mae: 17.7891 - val_loss: 404.8337 - val_mae: 18.0099
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 372.3109 - mae: 17.1659 - val_loss: 380.1979 - val_mae: 17.3433
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 347.9553 - mae: 16.4938 - val_loss: 354.7127 - val_mae: 16.6338
INFO:tensorflow:Assets written to: my_model_with_a_custom_parts/assets
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

model.get_config()
1

{'name': 'sequential_3',
 'layers': [{'class_name': 'InputLayer',
   'config': {'batch_input_shape': (None, 13),
    'dtype': 'float32',
    'sparse': False,
    'ragged': False,
    'name': 'dense_7_input'}},
  {'class_name': 'Dense',
   'config': {'name': 'dense_7',
    'trainable': True,
    'dtype': 'float32',
    'units': 30,
    'activation': 'relu',
    'use_bias': True,
    'kernel_initializer': {'class_name': 'HeNormal', 'config': {'seed': None}},
    'bias_initializer': {'class_name': 'Zeros', 'config': {}},
    'kernel_regularizer': None,
    'bias_regularizer': None,
    'activity_regularizer': None,
    'kernel_constraint': None,
    'bias_constraint': None}},
  {'class_name': 'Dense',
   'config': {'name': 'dense_8',
    'trainable': True,
    'dtype': 'float32',
    'units': 1,
    'activation': 'my_softplus',
    'use_bias': True,
    'kernel_initializer': 'my_glorot_initializer',
    'bias_initializer': {'class_name': 'Zeros', 'config': {}},
    'kernel_regularizer': 'my_l1_regularizer',
    'bias_regularizer': None,
    'activity_regularizer': None,
    'kernel_constraint': 'my_positive_weights',
    'bias_constraint': None}}]}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

model.layers[1].get_config()
1

{'name': 'dense_8',
 'trainable': True,
 'dtype': 'float32',
 'units': 1,
 'activation': 'my_softplus',
 'use_bias': True,
 'kernel_initializer': 'my_glorot_initializer',
 'bias_initializer': {'class_name': 'Zeros', 'config': {}},
 'kernel_regularizer': 'my_l1_regularizer',
 'bias_regularizer': None,
 'activity_regularizer': None,
 'kernel_constraint': 'my_positive_weights',
 'bias_constraint': None}
1
2
3
4
5
6
7
8
9
10
11
12
13

newmodel = tf.keras.models.load_model('my_model_with_a_custom_parts/')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2

Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 6ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 6ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
1
2
3
4
5
6
7
8
9
10

newmodel.layers[1].get_config()
1

{'name': 'dense_8',
 'trainable': True,
 'dtype': 'float32',
 'units': 1,
 'activation': 'my_softplus',
 'use_bias': True,
 'kernel_initializer': 'my_glorot_initializer',
 'bias_initializer': {'class_name': 'Zeros',
  'config': {},
  'shared_object_id': 4},
 'kernel_regularizer': 'my_l1_regularizer',
 'bias_regularizer': None,
 'activity_regularizer': None,
 'kernel_constraint': 'my_positive_weights',
 'bias_constraint': None}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

所以要判断模型是否一致,我们可以看一下他们的config

可以看到,对于这些自定义的函数无需人为指定也可以加载

newmodel1 = tf.keras.models.load_model('my_model_with_a_custom_parts/',custom_objects={"my_l1_regularizer":my_l1_regularizer,
                                                                                        "my_positive_weights":my_positive_weights,
                                                                                        "my_glorot_initializer":my_glorot_initializer,
                                                                                        "my_softplus":my_softplus
                                                                                        })
history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6

Epoch 1/5
13/13 [==============================] - 1s 17ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 7ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 7ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
1
2
3
4
5
6
7
8
9
10

这次可以看出,人为指定后,结果是一样的

2.2 类定义方式-正则

正则化也可以用类方式定义

class MyL1Regularizer(tf.keras.regularizers.Regularizer):
    def __init__(self,factor):
        self.factor=factor
    def __call__(self,weights):
        return tf.reduce_mean(tf.math.abs(self.factor*weights))
    def get_config(self):
        base_config = super().get_config()
        all_config={**base_config,'factor':self.factor}
1
2
3
4
5
6
7
8

tf.random.set_seed(42)
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),
    tf.keras.layers.Dense(1,activation=my_softplus,
                            kernel_initializer=my_glorot_initializer,
                            kernel_regularizer=MyL1Regularizer(0.01),
                            kernel_constraint=my_positive_weights
                            )
])
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
model.save('my_model_with_a_custom_parts1')
1
2
3
4
5
6
7
8
9
10
11
12

Epoch 1/10
13/13 [==============================] - 1s 19ms/step - loss: 515.4951 - mae: 20.7559 - val_loss: 531.4422 - val_mae: 21.1663
Epoch 2/10
13/13 [==============================] - 0s 5ms/step - loss: 501.5185 - mae: 20.4198 - val_loss: 519.0024 - val_mae: 20.8716
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 488.5792 - mae: 20.1095 - val_loss: 504.7497 - val_mae: 20.5305
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 473.7570 - mae: 19.7503 - val_loss: 488.5533 - val_mae: 20.1391
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 456.7914 - mae: 19.3408 - val_loss: 470.3438 - val_mae: 19.6946
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 437.9532 - mae: 18.8721 - val_loss: 450.0512 - val_mae: 19.1895
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 417.3983 - mae: 18.3609 - val_loss: 428.3052 - val_mae: 18.6335
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 395.6117 - mae: 17.7890 - val_loss: 404.7784 - val_mae: 18.0099
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 372.2539 - mae: 17.1659 - val_loss: 380.1387 - val_mae: 17.3433
Epoch 10/10
13/13 [==============================] - 0s 6ms/step - loss: 347.8943 - mae: 16.4938 - val_loss: 354.6496 - val_mae: 16.6338
INFO:tensorflow:Assets written to: my_model_with_a_custom_parts1/assets
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

newmodel = tf.keras.models.load_model('my_model_with_a_custom_parts1/')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2

Epoch 1/5
13/13 [==============================] - 1s 16ms/step - loss: 325.8748 - mae: 15.8555 - val_loss: 333.4524 - val_mae: 16.0323
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.9491 - mae: 15.2394 - val_loss: 312.9211 - val_mae: 15.4228
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 285.0223 - mae: 14.6250 - val_loss: 292.1203 - val_mae: 14.7955
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 264.9132 - mae: 13.9857 - val_loss: 271.5786 - val_mae: 14.1489
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 245.6193 - mae: 13.3311 - val_loss: 251.5693 - val_mae: 13.5434
1
2
3
4
5
6
7
8
9
10

newmodel1 = tf.keras.models.load_model('my_model_with_a_custom_parts1/',custom_objects={"MyL1Regularizer":MyL1Regularizer,
                                                                                        "my_positive_weights":my_positive_weights,
                                                                                        "my_glorot_initializer":my_glorot_initializer,
                                                                                        "my_softplus":my_softplus
                                                                                        })
history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6

2022-07-13 10:10:51.405105: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-07-13 10:10:52.073048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14637 MB memory:  -> device: 3, name: Tesla V100-PCIE-16GB-LS, pci bus id: 0000:dd:00.0, compute capability: 7.0



---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

/tmp/ipykernel_28741/247645023.py in 
      2                                                                                         "my_positive_weights":my_positive_weights,
      3                                                                                         "my_glorot_initializer":my_glorot_initializer,
----> 4                                                                                         "my_softplus":my_softplus
      5                                                                                         })
      6 history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/saving/saved_model/load.py in _revive_layer_or_model_from_config(self, metadata, node_id)
    536             'your class with `@keras.utils.register_keras_serializable` and '
    537             'include that file in your program, or pass your class in a '
--> 538             '`keras.utils.CustomObjectScope` that wraps this load call.') from e
    539       else:
    540         raise


RuntimeError: Unable to restore object of class 'Dense' likely due to name conflict with built-in Keras class ''. To override the built-in Keras definition of the object, decorate your class with `@keras.utils.register_keras_serializable` and include that file in your program, or pass your class in a `keras.utils.CustomObjectScope` that wraps this load call.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

这种情况,不知道为什么会报错,如果不指定反而不会报错,可能是因为第二个Dense已经是一个自定义层了需要做更多处理.

2.3 类定义方式-激活函数

对于激活函数

可以直接使用tf.keras.layers.Activation或tf.keras.layers.Layer来进行构建,因为有些激活函数带有要学习的参数,所以定义激活函数要像定义层一样,请参看下一节

2.4 类定义方式-初始化方式

接着使用类来定义初始化方法

class MyGlortInitializer(tf.keras.initializers.Initializer):
    def __call__(self,shape,dtype):
        self.stddev = tf.sqrt(2./(shape[0]+shape[1]))
        return tf.random.normal(shape,stddev=self.stddev,dtype=dtype) #正常还要写一个get_config
1
2
3
4

初始化没必要做到能保存，因为模型训练后或不训练，再次加载时用不到，只有第一次才能用到，所以get_config之类的，在initializer中是没有的

tf.random.set_seed(42)
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),
    tf.keras.layers.Dense(1,activation=my_softplus,
                            kernel_initializer=MyGlortInitializer,
                            kernel_regularizer=my_l1_regularizer,
                            kernel_constraint=my_positive_weights
                            )
])
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
model.save('my_model_with_a_custom_parts2')
1
2
3
4
5
6
7
8
9
10
11
12

Epoch 1/10
13/13 [==============================] - 1s 15ms/step - loss: 515.5219 - mae: 20.7559 - val_loss: 531.4690 - val_mae: 21.1663
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 501.5470 - mae: 20.4198 - val_loss: 519.0331 - val_mae: 20.8716
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 488.6118 - mae: 20.1095 - val_loss: 504.7845 - val_mae: 20.5305
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 473.7936 - mae: 19.7504 - val_loss: 488.5923 - val_mae: 20.1391
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 456.8321 - mae: 19.3409 - val_loss: 470.3868 - val_mae: 19.6946
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 437.9981 - mae: 18.8721 - val_loss: 450.0984 - val_mae: 19.1896
Epoch 7/10
13/13 [==============================] - 0s 6ms/step - loss: 417.4472 - mae: 18.3609 - val_loss: 428.3565 - val_mae: 18.6335
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 395.6647 - mae: 17.7891 - val_loss: 404.8337 - val_mae: 18.0099
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 372.3109 - mae: 17.1659 - val_loss: 380.1979 - val_mae: 17.3433
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 347.9553 - mae: 16.4938 - val_loss: 354.7128 - val_mae: 16.6338
INFO:tensorflow:Assets written to: my_model_with_a_custom_parts2/assets
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

newmodel = tf.keras.models.load_model('my_model_with_a_custom_parts2')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2

Epoch 1/5
13/13 [==============================] - 1s 14ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 6ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
1
2
3
4
5
6
7
8
9
10

newmodel1 = tf.keras.models.load_model('my_model_with_a_custom_parts2/',custom_objects={"my_l1_regularizer":my_l1_regularizer,
                                                                                        "my_positive_weights":my_positive_weights,
                                                                                        "MyGlorotInitizlizer":MyGlortInitializer,
                                                                                        "my_softplus":my_softplus
                                                                                        })
history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6

Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
1
2
3
4
5
6
7
8
9
10

2.5 类定义方式-Constraint

接着对于constraint这里就不写了,官方文档中写到,constraint是一种stateless的,对于这种,无__init__,无需get_config,只要重写__call__就完事

class NonNegative(tf.keras.constraints.Constraint):
    def __call__(self,w):
        return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)
    def get_config(self):
        base_config=super().get_config()
        return base_config
1
2
3
4
5
6

3 自定义层

exponential_layer=tf.keras.layers.Lambda(lambda x :tf.math.exp(x))
1

exponential_layer([-1.,0.,1.])
1

如果要预测的结果是正,且有很大并异的比例[0.001,0.1,10,1000] ,可以在最后一层加上指数函数

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),
    tf.keras.layers.Dense(1),
    exponential_layer
    ]
)
model.compile(loss='mse',optimizer='adam',metrics=["mae"]) 
history2=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6
7
8
9
10

Epoch 1/10
13/13 [==============================] - 0s 13ms/step - loss: 5567.9629 - mae: 32.7794 - val_loss: 6352.1050 - val_mae: 32.6988
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 1320.4006 - mae: 23.1720 - val_loss: 1681.2922 - val_mae: 24.9112
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 755.1310 - mae: 20.5381 - val_loss: 980.6192 - val_mae: 22.5717
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 538.5795 - mae: 19.2997 - val_loss: 816.2770 - val_mae: 21.7834
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 490.4356 - mae: 18.8144 - val_loss: 729.4755 - val_mae: 21.2690
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 463.7896 - mae: 18.5437 - val_loss: 675.5165 - val_mae: 20.8762
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 448.9651 - mae: 18.3412 - val_loss: 632.6336 - val_mae: 20.5424
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 431.8742 - mae: 18.1127 - val_loss: 606.0991 - val_mae: 20.2699
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 419.8213 - mae: 17.9123 - val_loss: 587.3072 - val_mae: 20.0439
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 410.5321 - mae: 17.7241 - val_loss: 565.5510 - val_mae: 19.7840
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

model.save('my_model_with_a_custom_layer1')
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_layer1/assets
1

newmodel = tf.keras.models.load_model('my_model_with_a_custom_layer1')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2

Epoch 1/5
13/13 [==============================] - 0s 14ms/step - loss: 385.0820 - mae: 17.2846 - val_loss: 453.2182 - val_mae: 18.4360
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 359.4727 - mae: 16.8214 - val_loss: 418.5759 - val_mae: 17.7304
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 344.2751 - mae: 16.4029 - val_loss: 402.5103 - val_mae: 17.4366
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 328.9195 - mae: 15.9526 - val_loss: 390.0414 - val_mae: 17.2147
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 312.5679 - mae: 15.4785 - val_loss: 363.0509 - val_mae: 16.6054
1
2
3
4
5
6
7
8
9
10

对于更复杂的自定义层

class MyDense(tf.keras.layers.Layer):
    def __init__(self,units,activation=None,**kwargs):
        super().__init__(**kwargs)
        self.units= units
        self.activation=tf.keras.activations.get(activation)
    def build(self,input_shape):
        self.kernel = self.add_weight(
            name='kernel',
            shape=[input_shape[-1],self.units],
            initializer='he_normal',
            trainable=True
        )
        self.bias = self.add_weight(
            name='bias',
            shape=[self.units],
            initializer='zeros',
            trainable=True
        )
        # super().build(input_shape)
    def call(self,x):
        return self.activation(x@self.kernel+self.bias)
    def get_config(self):
        base_config=super().get_config()
        all_config={**base_config,'units':self.units,'activation':tf.keras.activations.serialize(self.activation)}
        return all_config
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

看一下这个激活函数的操作

ac = tf.keras.activations.get('relu')
print(tf.keras.activations.serialize(ac))
1
2

relu
1

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    MyDense(30,activation='relu'),
    MyDense(1),
    ]
)
model.compile(loss='mse',optimizer='adam',metrics=["mae"]) 
history2=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6
7
8
9

Epoch 1/10
13/13 [==============================] - 0s 13ms/step - loss: 616.1592 - mae: 23.1205 - val_loss: 639.0024 - val_mae: 23.6284
Epoch 2/10
13/13 [==============================] - 0s 5ms/step - loss: 598.2690 - mae: 22.7416 - val_loss: 620.9468 - val_mae: 23.2488
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 580.7922 - mae: 22.3643 - val_loss: 603.4016 - val_mae: 22.8720
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 563.6730 - mae: 21.9861 - val_loss: 586.1691 - val_mae: 22.4956
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 546.5667 - mae: 21.6060 - val_loss: 569.0162 - val_mae: 22.1156
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 529.6218 - mae: 21.2165 - val_loss: 551.5815 - val_mae: 21.7227
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 512.6191 - mae: 20.8243 - val_loss: 534.1431 - val_mae: 21.3218
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 495.5235 - mae: 20.4124 - val_loss: 516.2220 - val_mae: 20.8990
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 477.9213 - mae: 19.9843 - val_loss: 497.9936 - val_mae: 20.4632
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 459.9037 - mae: 19.5355 - val_loss: 479.3126 - val_mae: 20.0070
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

model.save('my_model_with_a_custom_layer2')
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_layer2/assets
1

newmodel = tf.keras.models.load_model('my_model_with_a_custom_layer2')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2

Epoch 1/5
13/13 [==============================] - 1s 13ms/step - loss: 441.9317 - mae: 19.0821 - val_loss: 461.3201 - val_mae: 19.5506
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 424.6645 - mae: 18.6328 - val_loss: 443.4474 - val_mae: 19.0898
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 407.4524 - mae: 18.1880 - val_loss: 425.7647 - val_mae: 18.6200
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 390.4624 - mae: 17.7256 - val_loss: 408.2603 - val_mae: 18.1513
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 373.4209 - mae: 17.2619 - val_loss: 390.7575 - val_mae: 17.6752
1
2
3
4
5
6
7
8
9
10

接着看一个多输入多输出的层

class MyMultiLayer(tf.keras.layers.Layer):
    def call(self,x):
        x1,x2=x
        print("x1.shape:",x1.shape,"x2.shape:",x2.shape)
        return x1+x2,x1*x2,x1/x2
1
2
3
4
5

inputs1 = tf.keras.layers.Input(shape=[2])
inputs2 = tf.keras.layers.Input(shape=[2])
MyMultiLayer()((inputs1,inputs2))
1
2
3

x1.shape: (None, 2) x2.shape: (None, 2)





(,
 ,
 )
1
2
3
4
5
6
7
8
9

以上只是输放Placeholder的输入,当然也可以输入实际数值

x1,x2=np.array([[3.,6],[2.,7]]),np.array([[6.,12],[4.,3]])
MyMultiLayer()((x1,x2))
1
2

x1.shape: (2, 2) x2.shape: (2, 2)





(,
 ,
 )
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

接着创建一个在训练和推理过程中表现不一样的层

class MyGaussianNoise(tf.keras.layers.Layer):
    def __init__(self,stddev,**kwargs):
        super().__init__(**kwargs)
        self.stddev = stddev
    def call(self,x,training=None):
        if training:
            noise = tf.random.normal(tf.shape(x),stddev=self.stddev)
            return x+noise
        else:
            return x
    def get_config(self):
        base_config = super().get_config()
        all_config={**base_config,'stddev':self.stddev}
        return all_config
1
2
3
4
5
6
7
8
9
10
11
12
13
14

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    MyGaussianNoise(1.0,input_shape=input_shape),
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),
    tf.keras.layers.Dense(1),
    ]
)
model.compile(loss='mse',optimizer='adam',metrics=["mae"]) 
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
1
2
3
4
5
6
7
8
9
10

Epoch 1/10
13/13 [==============================] - 1s 14ms/step - loss: 468.3629 - mae: 19.6780 - val_loss: 516.2980 - val_mae: 20.8568
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 440.2289 - mae: 19.0264 - val_loss: 499.2351 - val_mae: 20.4551
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 427.3663 - mae: 18.7168 - val_loss: 482.1429 - val_mae: 20.0414
Epoch 4/10
13/13 [==============================] - 0s 6ms/step - loss: 410.8777 - mae: 18.2924 - val_loss: 465.1442 - val_mae: 19.6237
Epoch 5/10
13/13 [==============================] - 0s 6ms/step - loss: 388.5227 - mae: 17.6104 - val_loss: 448.0427 - val_mae: 19.1952
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 372.6901 - mae: 17.1882 - val_loss: 430.6292 - val_mae: 18.7507
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 357.3874 - mae: 16.8031 - val_loss: 413.2674 - val_mae: 18.2961
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 338.7885 - mae: 16.2308 - val_loss: 395.8601 - val_mae: 17.8260
Epoch 9/10
13/13 [==============================] - 0s 6ms/step - loss: 319.7017 - mae: 15.6614 - val_loss: 378.3233 - val_mae: 17.3410
Epoch 10/10
13/13 [==============================] - 0s 6ms/step - loss: 300.2906 - mae: 15.0687 - val_loss: 360.9408 - val_mae: 16.8501
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

model.save('my_model_with_a_custom_layer3/')
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_layer3/assets
1

model.layers[0].get_config()
1

{'name': 'my_gaussian_noise_4',
 'trainable': True,
 'batch_input_shape': (None, 13),
 'dtype': 'float32',
 'stddev': 1.0}
1
2
3
4
5

newmodel = tf.keras.models.load_model('my_model_with_a_custom_layer3')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
1
2

Epoch 1/5
13/13 [==============================] - 0s 14ms/step - loss: 278.7925 - mae: 14.3733 - val_loss: 343.0135 - val_mae: 16.3244
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 254.7551 - mae: 13.5867 - val_loss: 325.6881 - val_mae: 15.7965
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 243.1024 - mae: 13.2733 - val_loss: 308.8557 - val_mae: 15.2673
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 231.9742 - mae: 12.8450 - val_loss: 292.6225 - val_mae: 14.7712
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 215.9254 - mae: 12.1293 - val_loss: 276.9524 - val_mae: 14.3115
1
2
3
4
5
6
7
8
9
10

newmodel.layers[0].get_config()
1

{'name': 'my_gaussian_noise_4',
 'trainable': True,
 'batch_input_shape': (None, 13),
 'dtype': 'float32',
 'stddev': 1.0}
1
2
3
4
5

可以看出,模型实现原模型加载

4 自定义Metrics

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),
    tf.keras.layers.Dense(1)
    ]
)
model.compile(loss="mse",optimizer='adam',metrics=[create_huber(2.0)])  
history2=model.fit(x_train_scaled,y_train,epochs=3)
1
2
3
4
5
6
7
8
9

2022-07-26 10:29:30.486789: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-07-26 10:29:31.131913: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14637 MB memory:  -> device: 3, name: Tesla V100-PCIE-16GB-LS, pci bus id: 0000:dd:00.0, compute capability: 7.0


Epoch 1/3
13/13 [==============================] - 3s 4ms/step - loss: 491.9991 - huber_fn: 38.6896
Epoch 2/3
13/13 [==============================] - 0s 4ms/step - loss: 474.2117 - huber_fn: 37.8320
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 456.2298 - huber_fn: 36.9550
1
2
3
4
5
6
7
8
9
10
11

如果损失函数和评价指标使用同一个函数，你可能会看到结果有一些略微的不同.这是因为计算的操作顺序不完全相同，所以可能存以微小的错误。特别是如何使用 sample weights和class weights那么，结果将更加不一样。

fit()方法会一直跟踪记录所有batch的平均的损失，从训练开始起就这样。每个batch的损失值是由每个样本的损失，如果这个过程有样本或分类加权，会算上去的，求和然后除以样本数得来的。
而评价指标是从训练开始，记录所有单个样本的损失，然后然后算平均，再加权算

precision = tf.keras.metrics.Precision()
precision([0,1,1,1,0,1,0,1],[1,1,0,1,0,1,0,1])
1
2

precision([0,1,0,0,1,0,1,1],[1,0,1,1,0,0,0,0])
1

precision.variables
1

[,
 ]
1
2

precision.reset_states()
1

接着改一下

class HuberMetric(tf.keras.metrics.Metric):
    def __init__(self,threshold=1.0,**kwargs):
        super().__init__(**kwargs)
        self.threshold = threshold
        self.huber_fn = create_huber(threshold)
        self.total = self.add_weight("total",initializer="zeros")
        self.count = self.add_weight("count",initializer="zeros")
    def update_state(self,y_true,y_pred,sample_weight=None):
        sample_metric = self.huber_fn(y_true,y_pred)
        self.total.assign_add(tf.reduce_sum(sample_metric))
        self.count.assign_add(tf.cast(tf.size(y_true),tf.float32))
    def result(self):
        return self.total/self.count
    def get_config(self):
        base_config = super().get_config()
        all_config = {**base_config,'threshold':self.threshold}
        return all_config
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

m = HuberMetric(2.0)
# total = 2*|2-10|-2^2/2=14.0
# count = 1
# result = 14.0/1.0=14.0
m(tf.constant([2.0]),tf.constant([10.0]))
1
2
3
4
5

# total =total+ (|1.0-0|^2)/2 + (2*|5-9.25|-2^2/2)=14.0+7.0=21.0
# count = count +2=3
# result = total/count=21/3=7
m(tf.constant([[0.],[5]]),tf.constant([[1.],[9.25]]))
1
2
3
4

m.result()
1

m.variables
1

[,
 ]
1
2

m.reset_states()
m.variables
1
2

[,
 ]
1
2

检查我们自定义的hubermetric运行良好

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),
    tf.keras.layers.Dense(1)
    ]
)
model.compile(loss=create_huber(2.0),optimizer='adam',metrics=[HuberMetric(2.0)])  
history=model.fit(x_train_scaled,y_train,epochs=3)
1
2
3
4
5
6
7
8
9

Epoch 1/3
13/13 [==============================] - 0s 3ms/step - loss: 38.6856 - huber_metric_6: 38.6856
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 37.8080 - huber_metric_6: 37.8080
Epoch 3/3
13/13 [==============================] - 0s 2ms/step - loss: 36.9100 - huber_metric_6: 36.9100
1
2
3
4
5
6

model.save("my_model_with_a_custom_metric")
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_metric/assets
1

new_model = tf.keras.models.load_model("my_model_with_a_custom_metric/")
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
1
2

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/tmp/ipykernel_21219/1845581953.py in 
----> 1 new_model = tf.keras.models.load_model("my_model_with_a_custom_metric/")
      2 history1=newmodel.fit(x_train_scaled,y_train,epochs=3)


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/saving/saved_model/load.py in revive_custom_object(identifier, metadata)
    993   else:
    994     raise ValueError(
--> 995         f'Unable to restore custom object of type {identifier}. '
    996         f'Please make sure that any custom layers are included in the '
    997         f'`custom_objects` arg when calling `load_model()` and make sure that '


ValueError: Unable to restore custom object of type _tf_keras_metric. Please make sure that any custom layers are included in the `custom_objects` arg when calling `load_model()` and make sure that all layers implement `get_config` and `from_config`.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

new_model = tf.keras.models.load_model("my_model_with_a_custom_metric/",custom_objects={'huber_fn':create_huber(2.0),'HuberMetric':HuberMetric})
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
1
2

Epoch 1/3
13/13 [==============================] - 1s 3ms/step - loss: 35.9728 - huber_metric_6: 35.9728
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 35.0231 - huber_metric_6: 35.0231
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 34.0567 - huber_metric_6: 34.0567
1
2
3
4
5
6

model.metrics包括损失和评价指标，所以model.meterics[-1]是我们自定义的hubermetric

model.metrics
1

[,
 <__main__.HuberMetric at 0x7fb95c7a2f90>]
1
2

tf.keras.metrics.serialize(model.metrics[-1])
1

{'class_name': 'HuberMetric',
 'config': {'name': 'huber_metric_6', 'dtype': 'float32', 'threshold': 2.0}}
1
2

这个类的更简单的定义方式可以是：


class HuberMetric(tf.keras.metrics.Mean):
    def __init__(self,threshold=1.0,name='HuberMetric',dtype=None):
        super().__init__(name='HuberMetric',dtype=None)
        self.threshold = threshold
        self.huber_fn = create_huber(threshold)

    def update_state(self,y_true,y_pred,sample_weight=None):
        sample_metric = self.huber_fn(y_true,y_pred)
        super(HuberMetric,self).update_state(sample_metric,sample_weight)

    def get_config(self):
        base_config = super().get_config()
        all_config = {**base_config,'threshold':self.threshold}
        return all_config
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

这种定义方式可以更好的处理输入形状和sample weights

tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([
    tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),
    tf.keras.layers.Dense(1)
    ]
)
model.compile(loss=tf.keras.losses.Huber(2.0),optimizer='adam',weighted_metrics=[HuberMetric(2.0)])  

1
2
3
4
5
6
7
8
9

np.random.seed(42)
sample_weight = np.random.rand(len(y_train))
history=model.fit(x_train_scaled,y_train,epochs=3,sample_weight=sample_weight)
1
2
3

Epoch 1/3
13/13 [==============================] - 1s 4ms/step - loss: 19.2427 - HuberMetric: 38.8675
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 18.8211 - HuberMetric: 38.0161
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 18.3968 - HuberMetric: 37.1590
1
2
3
4
5
6

(history.history['loss'][0],history.history['HuberMetric'][0]*sample_weight.mean())
1

(19.242660522460938, 19.242658563073878)
1

model.save("my_model_with_a_custom_metric2")
1

INFO:tensorflow:Assets written to: my_model_with_a_custom_metric2/assets
1

new_model = tf.keras.models.load_model("my_model_with_a_custom_metric2/")
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
1
2

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/tmp/ipykernel_20639/4045318839.py in 
----> 1 new_model = tf.keras.models.load_model("my_model_with_a_custom_metric2/")
      2 history1=new_model.fit(x_train_scaled,y_train,epochs=3)


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/saving/saved_model/load.py in revive_custom_object(identifier, metadata)
    993   else:
    994     raise ValueError(
--> 995         f'Unable to restore custom object of type {identifier}. '
    996         f'Please make sure that any custom layers are included in the '
    997         f'`custom_objects` arg when calling `load_model()` and make sure that '


ValueError: Unable to restore custom object of type _tf_keras_metric. Please make sure that any custom layers are included in the `custom_objects` arg when calling `load_model()` and make sure that all layers implement `get_config` and `from_config`.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

new_model = tf.keras.models.load_model("my_model_with_a_custom_metric2/",custom_objects={"HuberMetric":HuberMetric})
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
1
2

Epoch 1/3
13/13 [==============================] - 0s 3ms/step - loss: 36.0534 - HuberMetric: 36.0534
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 35.1053 - HuberMetric: 35.1053
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 34.1408 - HuberMetric: 34.1408
1
2
3
4
5
6

5 关于serialize deserialize get register_keras_serializable的使用

这三个函数的使用还有一个要配合的是tf.keras.utils.register_keras_serializable.
通过一些代码我们来研究一下这三个函数的作用.
其中register_keras_serializable相当重要，可能使用户在使用tf.keras.models.load_model(custom_objects={…}),custom_objects不用由用户额外提供。也看到custom_objects是一个字典，key是我们自定义函数的名称。

5.1 serialize

maxnorm = tf.keras.constraints.MaxNorm(3,0)
output=tf.keras.constraints.serialize(maxnorm)
print(f"type:{type(output)} content:{output}")
print("get_config:",maxnorm.get_config())
1
2
3
4

type: content:{'class_name': 'MaxNorm', 'config': {'max_value': 3, 'axis': 0}}
get_config: {'max_value': 3, 'axis': 0}
1
2

可以看到serialize是比get_config反回的内空还要多，其中class_name的内容就是custom_object字典的key,但自定义的层的class_name就是自定义的函数或类的名称，关于类的查看5.4节

def relu_my(input):
    return tf.nn.relu(input)
output1 = tf.keras.constraints.serialize(relu_my)
output2 = tf.keras.activations.serialize(relu_my)
print(f"output1:{output1} output2:{output2}")
1
2
3
4
5

output1:relu_my output2:relu_my
1

5.2 deserialize

这个是个serialize相反的一对，可以从serialize的结果中反向获得要定义的对像，大致是这么个意思

maxnorm1=tf.keras.constraints.deserialize(output)
1

print(maxnorm1.get_config())
1

{'max_value': 3, 'axis': 0}
1

maxnorm1是和maxnorm有相同参数的一个新的层，也就是说我们可以能过deserialize重新获得这个层的定义

5.3 get

get是为了使用字符串来获得要定义的层，比如maxnorm的serialize的output,可以看到class_name

maxnorm2 = tf.keras.constraints.get('MaxNorm')
1

print(maxnorm2.get_config())
1

{'max_value': 2, 'axis': 0}
1

maxnorm2是一个拥有初始化参数的层，所以get的作用有限

5.4 register_keras_serializable

对比使用和不使用，及使用不同方式时的结果.首先查看不注册的情况下：

class NonNegative(tf.keras.constraints.Constraint):
    def __call__(self,w):
        return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)
    def get_config(self):
        base_config=super().get_config()
        return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
1
2
3
4
5
6
7
8
9

{'class_name': 'NonNegative', 'config': {}}
1

print(tf.keras.constraints.get('NonNegative'))
1

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/tmp/ipykernel_19639/1015723810.py in 
----> 1 print(tf.keras.constraints.get('NonNegative'))


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/constraints.py in get(identifier)
    341   elif isinstance(identifier, str):
    342     config = {'class_name': str(identifier), 'config': {}}
--> 343     return deserialize(config)
    344   elif callable(identifier):
    345     return identifier


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/constraints.py in deserialize(config, custom_objects)
    329       module_objects=globals(),
    330       custom_objects=custom_objects,
--> 331       printable_module_name='constraint')
    332 
    333 


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    663     config = identifier
    664     (cls, cls_config) = class_and_config_for_serialized_keras_object(
--> 665         config, module_objects, custom_objects, printable_module_name)
    666 
    667     # If this object has already been loaded (i.e. it's shared between multiple


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)
    561   if cls is None:
    562     raise ValueError(
--> 563         f'Unknown {printable_module_name}: {class_name}. Please ensure this '
    564         'object is passed to the `custom_objects` argument. See '
    565         'https://www.tensorflow.org/guide/keras/save_and_serialize'


ValueError: Unknown constraint: NonNegative. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

可以看到无法使用get这个方法

print(tf.keras.constraints.deserialize(out))
1

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

/tmp/ipykernel_19639/3405952053.py in 
----> 1 print(tf.keras.constraints.deserialize(out))


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/constraints.py in deserialize(config, custom_objects)
    329       module_objects=globals(),
    330       custom_objects=custom_objects,
--> 331       printable_module_name='constraint')
    332 
    333 


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    663     config = identifier
    664     (cls, cls_config) = class_and_config_for_serialized_keras_object(
--> 665         config, module_objects, custom_objects, printable_module_name)
    666 
    667     # If this object has already been loaded (i.e. it's shared between multiple


~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)
    561   if cls is None:
    562     raise ValueError(
--> 563         f'Unknown {printable_module_name}: {class_name}. Please ensure this '
    564         'object is passed to the `custom_objects` argument. See '
    565         'https://www.tensorflow.org/guide/keras/save_and_serialize'


ValueError: Unknown constraint: NonNegative. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

也无法使用deserialize。要想使用，需要register_keras_serializable

接着进行使用注册

@tf.keras.utils.register_keras_serializable()
class NonNegative(tf.keras.constraints.Constraint):
    def __call__(self,w):
        return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)
    def get_config(self):
        base_config=super().get_config()
        return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
1
2
3
4
5
6
7
8
9
10

{'class_name': 'Custom>NonNegative', 'config': {}}
1

tf.keras.utils.register_keras_serializable()里边有两个参数，我们接着看,其中package默认参数是Custom

@tf.keras.utils.register_keras_serializable(package="cc",name=None)
class NonNegative(tf.keras.constraints.Constraint):
    def __call__(self,w):
        return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)
    def get_config(self):
        base_config=super().get_config()
        return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
1
2
3
4
5
6
7
8
9
10

{'class_name': 'cc>NonNegative', 'config': {}}
1

@tf.keras.utils.register_keras_serializable(name="ccc")
class NonNegative(tf.keras.constraints.Constraint):
    def __call__(self,w):
        return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)
    def get_config(self):
        base_config=super().get_config()
        return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
1
2
3
4
5
6
7
8
9
10

{'class_name': 'Custom>ccc', 'config': {}}
1

@tf.keras.utils.register_keras_serializable(package='',name="dd")
class NonNegative(tf.keras.constraints.Constraint):
    def __call__(self,w):
        return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)
    def get_config(self):
        base_config=super().get_config()
        return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
1
2
3
4
5
6
7
8
9
10

{'class_name': '>dd', 'config': {}}
1

@tf.keras.utils.register_keras_serializable(package='ok',name="dd")
class NonNegative(tf.keras.constraints.Constraint):
    def __call__(self,w):
        return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)
    def get_config(self):
        base_config=super().get_config()
        return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
1
2
3
4
5
6
7
8
9
10

{'class_name': 'ok>dd', 'config': {}}
1

print(tf.keras.constraints.get('ok>dd'))
print(non)
print(tf.keras.constraints.deserialize(out))
1
2
3

<__main__.NonNegative object at 0x7fcfcf681dd0>
<__main__.NonNegative object at 0x7fd08328d2d0>
<__main__.NonNegative object at 0x7fcfcf6813d0>
1
2
3

总结来说就是package name两个参数分别决定了class_name中’>'这个符号的前边和后边部分，而使用register_keras_serializable可以方便用户在使用自定义的东西时，不需要在加载模型时tf.keras.models.load_model 中传入custom_objects,这对模型部署是极不安全和便利的。以上1到4节部分的内容,都可以加上这个，就不用写custom_objects了

有多种serialize,deserialize,get的组合：tf.keras.layers,tf.keras.initializers,tf.keras.regularizers,tf.keras.losses,tf.keras.constriants等，具体的要上官方文档查看一下

import tensorflow as tf
1

2022-07-28 20:38:53.365900: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
1

对于其它普通的类或函数也可以进行序列化

@tf.keras.utils.register_keras_serializable(package="test")
class TestSample:
    def __call__(self,shape=None,dtype='float32'):
        print("test")
    def get_config(self):
        return{}
1
2
3
4
5
6

test= TestSample()
print(test())
print(tf.keras.utils.serialize_keras_object(test))
1
2
3

test
None
{'class_name': 'test>TestSample', 'config': {}}
1
2
3

@tf.keras.utils.register_keras_serializable(package='test1')
def test(shape=None):
    print("test1")
    return None
1
2
3
4



print(tf.keras.utils.serialize_keras_object(test))
1
2
3

test1>test
1

相关阅读:
Eigen中三维位姿表示方式以及相互转换
 Java中可以用的大数据推荐算法
 基于ssm+vue的班级同学录网站管理系统 elementui
安装YMFE/yapi API管理服务器（Ubuntu20）
C#Winform 打开文件浏览器
 ROS系列：第六章机器人建模
 Django 解决 CSRF 问题
 R语言layout () 函数
 全国双非院校考研信息汇总整理 Part.5
密码学系列之:PEM和PKCS7,PKCS8,PKCS12
原文地址：https://blog.csdn.net/u011119817/article/details/126054211