--- 본 포스팅은 데이콘 서포터즈 "데이크루 2기" 활동의 일환입니다 ---
- 안녕하세요 데이콘 서포터즈 데이크루 2기 포스(POS)팀의 Rahites입니다 :)
- POS팀은 Python OpenCV Study의 약자로 활동 기간동안 저희 팀은 '파이썬으로 만드는 OpenCV 프로젝트' 책을 가지고 OpenCV를 공부해보고 프로젝트를 진행할 것입니다.
- 자세한 스터디 계획과 운영 방안은 아래의 포스팅에서 확인하실 수 있습니다.
https://dacon.io/codeshare/4759?utm_source=dacrew&utm_medium=432727&utm_campaign=dacrew_2
[🔥포스🔥] 0편. OpenCV 예고장
dacon.io
https://dacon.io/codeshare/4956?utm_source=dacrew&utm_medium=432727&utm_campaign=dacrew_2
[🔥포스🔥] 첫번째 프로젝트, Multi-Hand Gesture Recognition
dacon.io
# 오늘은!! 포스팀이 만든 첫번째 프로젝트를 발표해보도록 하겠습니다!!
주제는!!!
✨ Multi Hand Gesture Reconition (두 손 동작 인식) ✨
으로 목차는 다음과 같습니다.
1. 개요
2. 데이터 설명
3. 모형 설명
4. 코드
5. 과적합 방지 기법
5.1. Dropout
5.2. Batch Normalization
5.3. Early Stopping
5.4. L1, L2 규제
6. 최종 모형
7. 결론
** 그럼 시작합니다~!! (≧∇≦)ﾉ **
1. 개요¶
코로나로 인해 비대면으로 진행되는 업무가 많이 생겼습니다. 가령 수업, 미팅 등 입니다. 이렇게 비대면 수요가 높아지다보니 영상에서 어떤 task를 이용한 사업들도 많이 나왔습니다. 최근 기사에서는 '비대면 술자리' 라는 콘텐츠로 화면에서 물체를 분류하거나 처리하여 게임에 적용하는 등 많은 사례가 나오고 있습니다. 이에, 동작을 인식하는 프로젝트를 하면서 실제 현업에서 적용된 사례를 간단하게나마 구현해보고 더 나아가 모든 비대면 업무 참가자들이 마이크나 채팅을 통해 대답하는 것보다 손 동작을 인식함으로써 효율적인 의사소통을 기대하는 마음으로 프로젝트를 진행하였습니다.
2. 데이터 소개¶
분류하고자 하는 손 동작 인식은 두 손을 이용한 동작 인식입니다. 사용한 동작은 총 4가지 입니다.
- yes : 두 손을 동그랗게 모아 o를 나타내는 동작입니다.
- no : 두 손을 cross하여 x를 나타내는 동작입니다.
- like : 두 손으로 엄지척을 하는 동작입니다.
- heart : 두 손으로 하트를 그리는 동작입니다.
3. 모형 설명¶
Base Model로 딥러닝 중 LSTM(Long Short Term Memory)을 사용하였습니다. 장/단기 기억을 가능하게 설계한 신경망 구조를 의미하는 LSTM은 주로 시계열 처리나 자연어 처리에 많이 사용되곤 합니다. 여기에서는 동작의 움직임을 frame 당으로 바꾸어 vector로 만들어 input으로 들어갑니다.
4. 코드¶
데이터 합치기¶
위 단계에서는 yes, no, like, heart인 4가지 동작 각각의 인식을 담은 배열 npy 파일을 합치려고 합니다.
In [ ]:
data_yes = np.load('dataset/seq_yes_1652512945.npy')
data_no = np.load('dataset/seq_no_1652512945.npy')
data_like = np.load('dataset/seq_like_1652512945.npy')
data_heart = np.load('dataset/seq_heart_1652512945.npy')
print(data_yes.shape)
print(data_no.shape)
print(data_like.shape)
print(data_heart.shape)
(1006, 10, 111) (770, 10, 111) (1073, 10, 111) (1143, 10, 111)
In [ ]:
actions = ['yes', 'no', 'like', 'heart']
data = np.concatenate([
np.load('dataset/seq_yes_1652512945.npy'),
np.load('dataset/seq_no_1652512945.npy'),
np.load('dataset/seq_like_1652512945.npy'),
np.load('dataset/seq_heart_1652512945.npy'),
], axis=0)
data.shape
Out[ ]:
(3992, 10, 111)
데이터 x, y 나누기¶
In [ ]:
x_data = data[:, :, :-1]
labels = data[:, 0, -1]
print(x_data.shape)
print(labels.shape)
(3992, 10, 110) (3992,)
In [ ]:
np.unique(labels)
Out[ ]:
array([0., 1., 2., 3.])
In [ ]:
from tensorflow.keras.utils import to_categorical
y_data = to_categorical(labels, num_classes=len(actions))
y_data.shape
Out[ ]:
(3992, 4)
X,y train_test_split¶
In [ ]:
from sklearn.model_selection import train_test_split
np.random.seed(1)
x_data = x_data.astype(np.float32)
y_data = y_data.astype(np.float32)
x_train, x_val, y_train, y_val = train_test_split(x_data, y_data, test_size=0.25, random_state=42)
print(x_train.shape, y_train.shape)
print(x_val.shape, y_val.shape)
(2994, 10, 110) (2994, 4) (998, 10, 110) (998, 4)
판단 척도¶
판단 척도는 F1 score를 사용하였습니다. F1 score는 정밀도와 재현율을 결합한 지표로 정밀도와 재현율이 어느 한 쪽으로 치우치지 않을 때 높은 값을 나타냅니다.
In [ ]:
import tensorflow as tf
def metric_F1score(y_true,y_pred):
TP=tf.reduce_sum(y_true*tf.round(y_pred))
TN=tf.reduce_sum((1-y_true)*(1-tf.round(y_pred)))
FP=tf.reduce_sum((1-y_true)*tf.round(y_pred))
FN=tf.reduce_sum(y_true*(1-tf.round(y_pred)))
precision=TP/(TP+FP)
recall=TP/(TP+FN)
F1score=2*precision*recall/(precision+recall)
return F1score
Modeling¶
In [ ]:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout
model = Sequential([
LSTM(64, activation='relu', input_shape=x_train.shape[1:3]),
Dense(32, activation='relu'),
Dense(len(actions), activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=[metric_F1score])
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm (LSTM) (None, 64) 44800 _________________________________________________________________ dense (Dense) (None, 32) 2080 _________________________________________________________________ dense_1 (Dense) (None, 4) 132 ================================================================= Total params: 47,012 Trainable params: 47,012 Non-trainable params: 0 _________________________________________________________________
Training¶
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
history = model.fit(
x_train,
y_train,
validation_data=(x_val, y_val),
epochs=50,
callbacks=[
ModelCheckpoint('models/multi_hand_gesture_classifier.h5', monitor='val_metric_F1score', verbose=1, save_best_only=True, mode='auto'),
ReduceLROnPlateau(monitor='val_metric_F1score', factor=0.5, patience=50, verbose=1, mode='auto')
]
)
Epoch 1/50 94/94 [==============================] - 5s 39ms/step - loss: 19.8059 - metric_F1score: 0.4988 - val_loss: 1.9830 - val_metric_F1score: 0.8410 Epoch 00001: val_metric_F1score improved from inf to 0.84099, saving model to models\multi_hand_gesture_classifier.h5 Epoch 2/50 94/94 [==============================] - 1s 9ms/step - loss: 2.4794 - metric_F1score: 0.8664 - val_loss: 1.8825 - val_metric_F1score: 0.8697 Epoch 00002: val_metric_F1score did not improve from 0.84099 Epoch 3/50 94/94 [==============================] - 1s 8ms/step - loss: 1.5710 - metric_F1score: 0.8912 - val_loss: 0.4678 - val_metric_F1score: 0.9658 Epoch 00003: val_metric_F1score did not improve from 0.84099 Epoch 4/50 94/94 [==============================] - 1s 6ms/step - loss: 0.3725 - metric_F1score: 0.9751 - val_loss: 0.4184 - val_metric_F1score: 0.9707 Epoch 00004: val_metric_F1score did not improve from 0.84099 Epoch 5/50 94/94 [==============================] - 1s 8ms/step - loss: 0.3537 - metric_F1score: 0.9721 - val_loss: 0.5220 - val_metric_F1score: 0.9655 Epoch 00005: val_metric_F1score did not improve from 0.84099 Epoch 6/50 94/94 [==============================] - 1s 6ms/step - loss: 0.2746 - metric_F1score: 0.9819 - val_loss: 0.3065 - val_metric_F1score: 0.9795 Epoch 00006: val_metric_F1score did not improve from 0.84099 Epoch 7/50 94/94 [==============================] - 0s 5ms/step - loss: 0.1709 - metric_F1score: 0.9869 - val_loss: 0.2327 - val_metric_F1score: 0.9814 Epoch 00007: val_metric_F1score did not improve from 0.84099 Epoch 8/50 94/94 [==============================] - 1s 7ms/step - loss: 0.0763 - metric_F1score: 0.9879 - val_loss: 0.3532 - val_metric_F1score: 0.9746 Epoch 00008: val_metric_F1score did not improve from 0.84099 Epoch 9/50 94/94 [==============================] - 1s 6ms/step - loss: 0.1375 - metric_F1score: 0.9827 - val_loss: 0.0722 - val_metric_F1score: 0.9893 Epoch 00009: val_metric_F1score did not improve from 0.84099 Epoch 10/50 94/94 [==============================] - 1s 6ms/step - loss: 0.0264 - metric_F1score: 0.9961 - val_loss: 0.1912 - val_metric_F1score: 0.9736 Epoch 00010: val_metric_F1score did not improve from 0.84099 Epoch 11/50 94/94 [==============================] - 0s 5ms/step - loss: 0.3840 - metric_F1score: 0.9754 - val_loss: 0.2616 - val_metric_F1score: 0.9834 Epoch 00011: val_metric_F1score did not improve from 0.84099 Epoch 12/50 94/94 [==============================] - 0s 5ms/step - loss: 0.2023 - metric_F1score: 0.9894 - val_loss: 0.1719 - val_metric_F1score: 0.9863 Epoch 00012: val_metric_F1score did not improve from 0.84099 Epoch 13/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0743 - metric_F1score: 0.9912 - val_loss: 0.0504 - val_metric_F1score: 0.9922 Epoch 00013: val_metric_F1score did not improve from 0.84099 Epoch 14/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1051 - metric_F1score: 0.9900 - val_loss: 0.1839 - val_metric_F1score: 0.9782 Epoch 00014: val_metric_F1score did not improve from 0.84099 Epoch 15/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0773 - metric_F1score: 0.9907 - val_loss: 0.0643 - val_metric_F1score: 0.9902 Epoch 00015: val_metric_F1score did not improve from 0.84099 Epoch 16/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0277 - metric_F1score: 0.9953 - val_loss: 0.0612 - val_metric_F1score: 0.9912 Epoch 00016: val_metric_F1score did not improve from 0.84099 Epoch 17/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0013 - metric_F1score: 0.9997 - val_loss: 0.0812 - val_metric_F1score: 0.9922 Epoch 00017: val_metric_F1score did not improve from 0.84099 Epoch 18/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0203 - metric_F1score: 0.9967 - val_loss: 0.0897 - val_metric_F1score: 0.9912 Epoch 00018: val_metric_F1score did not improve from 0.84099 Epoch 19/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0063 - metric_F1score: 0.9985 - val_loss: 0.0668 - val_metric_F1score: 0.9922 Epoch 00019: val_metric_F1score did not improve from 0.84099 Epoch 20/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0011 - metric_F1score: 0.9997 - val_loss: 0.0976 - val_metric_F1score: 0.9912 Epoch 00020: val_metric_F1score did not improve from 0.84099 Epoch 21/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0121 - metric_F1score: 0.9981 - val_loss: 0.1075 - val_metric_F1score: 0.9932 Epoch 00021: val_metric_F1score did not improve from 0.84099 Epoch 22/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0424 - metric_F1score: 0.9959 - val_loss: 0.0270 - val_metric_F1score: 0.9961 Epoch 00022: val_metric_F1score did not improve from 0.84099 Epoch 23/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0248 - metric_F1score: 0.9974 - val_loss: 0.0431 - val_metric_F1score: 0.9951 Epoch 00023: val_metric_F1score did not improve from 0.84099 Epoch 24/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0038 - metric_F1score: 0.9991 - val_loss: 0.0213 - val_metric_F1score: 0.9951 Epoch 00024: val_metric_F1score did not improve from 0.84099 Epoch 25/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0017 - metric_F1score: 0.9995 - val_loss: 0.0126 - val_metric_F1score: 0.9990 Epoch 00025: val_metric_F1score did not improve from 0.84099 Epoch 26/50 94/94 [==============================] - 1s 6ms/step - loss: 8.7674e-05 - metric_F1score: 1.0000 - val_loss: 0.0126 - val_metric_F1score: 0.9990 Epoch 00026: val_metric_F1score did not improve from 0.84099 Epoch 27/50 94/94 [==============================] - 0s 5ms/step - loss: 3.6795e-05 - metric_F1score: 1.0000 - val_loss: 0.0126 - val_metric_F1score: 0.9990 Epoch 00027: val_metric_F1score did not improve from 0.84099 Epoch 28/50 94/94 [==============================] - 0s 4ms/step - loss: 3.1854e-05 - metric_F1score: 1.0000 - val_loss: 0.0131 - val_metric_F1score: 0.9980 Epoch 00028: val_metric_F1score did not improve from 0.84099 Epoch 29/50 94/94 [==============================] - 0s 4ms/step - loss: 3.3502e-05 - metric_F1score: 1.0000 - val_loss: 0.0133 - val_metric_F1score: 0.9980 Epoch 00029: val_metric_F1score did not improve from 0.84099 Epoch 30/50 94/94 [==============================] - 0s 4ms/step - loss: 5.8574e-05 - metric_F1score: 1.0000 - val_loss: 0.0130 - val_metric_F1score: 0.9990 Epoch 00030: val_metric_F1score did not improve from 0.84099 Epoch 31/50 94/94 [==============================] - 0s 4ms/step - loss: 4.8796e-05 - metric_F1score: 1.0000 - val_loss: 0.0129 - val_metric_F1score: 0.9990 Epoch 00031: val_metric_F1score did not improve from 0.84099 Epoch 32/50 94/94 [==============================] - 0s 5ms/step - loss: 4.5825e-05 - metric_F1score: 1.0000 - val_loss: 0.0130 - val_metric_F1score: 0.9990 Epoch 00032: val_metric_F1score did not improve from 0.84099 Epoch 33/50 94/94 [==============================] - 0s 5ms/step - loss: 2.7856e-05 - metric_F1score: 1.0000 - val_loss: 0.0133 - val_metric_F1score: 0.9980 Epoch 00033: val_metric_F1score did not improve from 0.84099 Epoch 34/50 94/94 [==============================] - 0s 5ms/step - loss: 5.9260e-05 - metric_F1score: 1.0000 - val_loss: 0.0132 - val_metric_F1score: 0.9980 Epoch 00034: val_metric_F1score did not improve from 0.84099 Epoch 35/50 94/94 [==============================] - 0s 4ms/step - loss: 1.4244e-05 - metric_F1score: 1.0000 - val_loss: 0.0134 - val_metric_F1score: 0.9980 Epoch 00035: val_metric_F1score did not improve from 0.84099 Epoch 36/50 94/94 [==============================] - 0s 4ms/step - loss: 3.1728e-05 - metric_F1score: 1.0000 - val_loss: 0.0136 - val_metric_F1score: 0.9980 Epoch 00036: val_metric_F1score did not improve from 0.84099 Epoch 37/50 94/94 [==============================] - 0s 4ms/step - loss: 2.2182e-05 - metric_F1score: 1.0000 - val_loss: 0.0137 - val_metric_F1score: 0.9971 Epoch 00037: val_metric_F1score did not improve from 0.84099 Epoch 38/50 94/94 [==============================] - 0s 4ms/step - loss: 2.4824e-05 - metric_F1score: 1.0000 - val_loss: 0.0136 - val_metric_F1score: 0.9980 Epoch 00038: val_metric_F1score did not improve from 0.84099 Epoch 39/50 94/94 [==============================] - 0s 4ms/step - loss: 1.3582e-05 - metric_F1score: 1.0000 - val_loss: 0.0134 - val_metric_F1score: 0.9980 Epoch 00039: val_metric_F1score did not improve from 0.84099 Epoch 40/50 94/94 [==============================] - 0s 4ms/step - loss: 2.7582e-05 - metric_F1score: 1.0000 - val_loss: 0.0134 - val_metric_F1score: 0.9980 Epoch 00040: val_metric_F1score did not improve from 0.84099 Epoch 41/50 94/94 [==============================] - 0s 4ms/step - loss: 2.6617e-05 - metric_F1score: 1.0000 - val_loss: 0.0134 - val_metric_F1score: 0.9980 Epoch 00041: val_metric_F1score did not improve from 0.84099 Epoch 42/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0350e-05 - metric_F1score: 1.0000 - val_loss: 0.0135 - val_metric_F1score: 0.9980 Epoch 00042: val_metric_F1score did not improve from 0.84099 Epoch 43/50 94/94 [==============================] - 0s 4ms/step - loss: 2.0126e-05 - metric_F1score: 1.0000 - val_loss: 0.0136 - val_metric_F1score: 0.9980 Epoch 00043: val_metric_F1score did not improve from 0.84099 Epoch 44/50 94/94 [==============================] - 0s 4ms/step - loss: 2.7411e-05 - metric_F1score: 1.0000 - val_loss: 0.0133 - val_metric_F1score: 0.9980 Epoch 00044: val_metric_F1score did not improve from 0.84099 Epoch 45/50 94/94 [==============================] - 0s 3ms/step - loss: 9.6485e-06 - metric_F1score: 1.0000 - val_loss: 0.0135 - val_metric_F1score: 0.9980 Epoch 00045: val_metric_F1score did not improve from 0.84099 Epoch 46/50 94/94 [==============================] - 0s 4ms/step - loss: 7.8296e-06 - metric_F1score: 1.0000 - val_loss: 0.0135 - val_metric_F1score: 0.9980 Epoch 00046: val_metric_F1score did not improve from 0.84099 Epoch 47/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0414e-05 - metric_F1score: 1.0000 - val_loss: 0.0136 - val_metric_F1score: 0.9980 Epoch 00047: val_metric_F1score did not improve from 0.84099 Epoch 48/50 94/94 [==============================] - 0s 5ms/step - loss: 9.0207e-06 - metric_F1score: 1.0000 - val_loss: 0.0135 - val_metric_F1score: 0.9980 Epoch 00048: val_metric_F1score did not improve from 0.84099 Epoch 49/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0663e-05 - metric_F1score: 1.0000 - val_loss: 0.0131 - val_metric_F1score: 0.9990 Epoch 00049: val_metric_F1score did not improve from 0.84099 Epoch 50/50 94/94 [==============================] - 0s 4ms/step - loss: 1.9218e-05 - metric_F1score: 1.0000 - val_loss: 0.0129 - val_metric_F1score: 0.9990 Epoch 00050: val_metric_F1score did not improve from 0.84099
Results Analysis¶
In [ ]:
import matplotlib.pyplot as plt
fig, loss_ax = plt.subplots(figsize=(16, 10))
acc_ax = loss_ax.twinx()
loss_ax.set_xlabel('epoch')
loss_ax.set_ylabel('loss')
acc_ax.plot(history.history['metric_F1score'], 'b', label='train f1')
acc_ax.plot(history.history['val_metric_F1score'], 'g', label='val f1')
acc_ax.set_ylabel('f1-score')
acc_ax.legend(loc='upper left')
plt.show()
In [ ]:
from sklearn.metrics import multilabel_confusion_matrix
from tensorflow.keras.models import load_model
y_pred = model.predict(x_val)
multilabel_confusion_matrix(np.argmax(y_val, axis=1), np.argmax(y_pred, axis=1))
Out[ ]:
array([[[730, 1],
[ 0, 267]],
[[797, 0],
[ 1, 200]],
[[731, 0],
[ 0, 267]],
[[735, 0],
[ 0, 263]]], dtype=int64)
결과를 살펴보면 엄청난 과적합이 일어난 것을 확인할 수 있습니다. 이를 과적합 방지 기법을 이용하여 살펴보겠습니다.
5. 과적합 방지 기법¶
5-1) Dropout¶
Dropout이란 서로 연결된 연결망에서 0부터 1사이의 확률로 선택적으로 노드를 제거하는 기법입니다.
좀 더 쉽게 설명하자면 너무 똑똑해지는 것을 방지하기 위해 (과적합방지) 인간처럼 기억을 잊어버릴 수 있게 한 것을 의미합니다.
Dropout을 적용하여 상관관계가 강한 Feature를 제외하고 학습해도 좋은 출력값을 얻을 수 있도록 최적화되었다면, 해당 Feature에만 출력값이 영향을 미치는 과대적합을 방지할 수 있습니다.
기존 모형에 이미 dropout 기법이 들어있지만 들어있는 계수의 값이 작다고 생각이 들어 값을 올려서 진행을 하도록 하겠습니다.
- tensorflow.layer.Dropout(rate, noise_shape=None, seed=None, kwargs)**
- rate : 0과 1 사이의 부동 소수점, 삭제할 입력 단위의 비율
- noise_shape : 입력과 곱할 바이너리 드롭아웃 마스크 모양을 나타내는 1D 정수 텐서
- seed : 임의의 seed
In [ ]:
# 수정 요망
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout
model2 = Sequential([
LSTM(64, activation='relu', input_shape=x_train.shape[1:3]),
Dropout(0.3),
Dense(32, activation='relu'),
Dropout(0.3),
Dense(len(actions), activation='softmax')
])
model2.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc', metric_F1score])
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
history = model2.fit(
x_train,
y_train,
validation_data=(x_val, y_val),
epochs=50,
callbacks=[ # callback? checkpoint?
ModelCheckpoint('models/multi_hand_gesture_classifier.h5', monitor='val_acc', verbose=1, save_best_only=True, mode='auto'),
ReduceLROnPlateau(monitor='val_metric_F1score', factor=0.5, patience=50, verbose=1, mode='auto')
]
)
Epoch 1/50 94/94 [==============================] - 2s 8ms/step - loss: 34.1904 - acc: 0.3890 - metric_F1score: 0.3876 - val_loss: 0.1916 - val_acc: 0.9649 - val_metric_F1score: 0.9627 Epoch 00001: val_acc improved from -inf to 0.96493, saving model to models\multi_hand_gesture_classifier.h5 Epoch 2/50 94/94 [==============================] - 0s 5ms/step - loss: 1.2209 - acc: 0.8059 - metric_F1score: 0.7983 - val_loss: 0.1046 - val_acc: 0.9709 - val_metric_F1score: 0.9646 Epoch 00002: val_acc improved from 0.96493 to 0.97094, saving model to models\multi_hand_gesture_classifier.h5 Epoch 3/50 94/94 [==============================] - 0s 4ms/step - loss: 0.6393 - acc: 0.8685 - metric_F1score: 0.8471 - val_loss: 0.1690 - val_acc: 0.9599 - val_metric_F1score: 0.9548 Epoch 00003: val_acc did not improve from 0.97094 Epoch 4/50 94/94 [==============================] - 0s 5ms/step - loss: 0.5815 - acc: 0.8698 - metric_F1score: 0.8529 - val_loss: 0.1001 - val_acc: 0.9760 - val_metric_F1score: 0.9619 Epoch 00004: val_acc improved from 0.97094 to 0.97595, saving model to models\multi_hand_gesture_classifier.h5 Epoch 5/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4369 - acc: 0.8947 - metric_F1score: 0.8650 - val_loss: 0.0450 - val_acc: 0.9860 - val_metric_F1score: 0.9839 Epoch 00005: val_acc improved from 0.97595 to 0.98597, saving model to models\multi_hand_gesture_classifier.h5 Epoch 6/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2291 - acc: 0.9471 - metric_F1score: 0.9224 - val_loss: 0.0250 - val_acc: 0.9910 - val_metric_F1score: 0.9870 Epoch 00006: val_acc improved from 0.98597 to 0.99098, saving model to models\multi_hand_gesture_classifier.h5 Epoch 7/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1385 - acc: 0.9719 - metric_F1score: 0.9524 - val_loss: 0.0219 - val_acc: 0.9940 - val_metric_F1score: 0.9951 Epoch 00007: val_acc improved from 0.99098 to 0.99399, saving model to models\multi_hand_gesture_classifier.h5 Epoch 8/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1342 - acc: 0.9665 - metric_F1score: 0.9518 - val_loss: 0.0075 - val_acc: 0.9980 - val_metric_F1score: 0.9976 Epoch 00008: val_acc improved from 0.99399 to 0.99800, saving model to models\multi_hand_gesture_classifier.h5 Epoch 9/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1545 - acc: 0.9663 - metric_F1score: 0.9538 - val_loss: 0.0472 - val_acc: 0.9910 - val_metric_F1score: 0.9917 Epoch 00009: val_acc did not improve from 0.99800 Epoch 10/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1349 - acc: 0.9735 - metric_F1score: 0.9643 - val_loss: 0.0076 - val_acc: 0.9980 - val_metric_F1score: 0.9976 Epoch 00010: val_acc did not improve from 0.99800 Epoch 11/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0896 - acc: 0.9702 - metric_F1score: 0.9668 - val_loss: 0.0068 - val_acc: 0.9980 - val_metric_F1score: 0.9971 Epoch 00011: val_acc did not improve from 0.99800 Epoch 12/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0746 - acc: 0.9756 - metric_F1score: 0.9752 - val_loss: 0.0060 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00012: val_acc did not improve from 0.99800 Epoch 13/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0703 - acc: 0.9838 - metric_F1score: 0.9826 - val_loss: 0.0023 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00013: val_acc improved from 0.99800 to 0.99900, saving model to models\multi_hand_gesture_classifier.h5 Epoch 14/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0719 - acc: 0.9880 - metric_F1score: 0.9864 - val_loss: 0.0102 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00014: val_acc did not improve from 0.99900 Epoch 15/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0708 - acc: 0.9879 - metric_F1score: 0.9867 - val_loss: 0.0014 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00015: val_acc improved from 0.99900 to 1.00000, saving model to models\multi_hand_gesture_classifier.h5 Epoch 16/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0494 - acc: 0.9912 - metric_F1score: 0.9912 - val_loss: 0.0021 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00016: val_acc did not improve from 1.00000 Epoch 17/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0622 - acc: 0.9864 - metric_F1score: 0.9843 - val_loss: 4.4725e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00017: val_acc did not improve from 1.00000 Epoch 18/50 94/94 [==============================] - 1s 6ms/step - loss: 0.0446 - acc: 0.9904 - metric_F1score: 0.9908 - val_loss: 3.6115e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00018: val_acc did not improve from 1.00000 Epoch 19/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0466 - acc: 0.9920 - metric_F1score: 0.9923 - val_loss: 3.7808e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00019: val_acc did not improve from 1.00000 Epoch 20/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0288 - acc: 0.9953 - metric_F1score: 0.9953 - val_loss: 2.7768e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00020: val_acc did not improve from 1.00000 Epoch 21/50 94/94 [==============================] - 1s 6ms/step - loss: 0.0229 - acc: 0.9947 - metric_F1score: 0.9953 - val_loss: 2.4419e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00021: val_acc did not improve from 1.00000 Epoch 22/50 94/94 [==============================] - 1s 6ms/step - loss: 0.0325 - acc: 0.9949 - metric_F1score: 0.9957 - val_loss: 9.2137e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00022: val_acc did not improve from 1.00000 Epoch 23/50 94/94 [==============================] - 1s 6ms/step - loss: 0.0242 - acc: 0.9950 - metric_F1score: 0.9952 - val_loss: 0.0021 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00023: val_acc did not improve from 1.00000 Epoch 24/50 94/94 [==============================] - 1s 6ms/step - loss: 0.0232 - acc: 0.9948 - metric_F1score: 0.9948 - val_loss: 2.1306e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00024: val_acc did not improve from 1.00000 Epoch 25/50 94/94 [==============================] - 1s 7ms/step - loss: 0.0266 - acc: 0.9944 - metric_F1score: 0.9940 - val_loss: 4.4663e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00025: val_acc did not improve from 1.00000 Epoch 26/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0294 - acc: 0.9952 - metric_F1score: 0.9949 - val_loss: 1.7144e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00026: val_acc did not improve from 1.00000 Epoch 27/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0174 - acc: 0.9982 - metric_F1score: 0.9982 - val_loss: 0.0010 - val_acc: 1.0000 - val_metric_F1score: 0.9995 Epoch 00027: val_acc did not improve from 1.00000 Epoch 28/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0397 - acc: 0.9936 - metric_F1score: 0.9938 - val_loss: 1.9972e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00028: val_acc did not improve from 1.00000 Epoch 29/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0155 - acc: 0.9958 - metric_F1score: 0.9963 - val_loss: 1.5378e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00029: val_acc did not improve from 1.00000 Epoch 30/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0166 - acc: 0.9962 - metric_F1score: 0.9963 - val_loss: 1.3054e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00030: val_acc did not improve from 1.00000 Epoch 31/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0171 - acc: 0.9975 - metric_F1score: 0.9975 - val_loss: 1.1474e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00031: val_acc did not improve from 1.00000 Epoch 32/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0173 - acc: 0.9986 - metric_F1score: 0.9987 - val_loss: 9.3023e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00032: val_acc did not improve from 1.00000 Epoch 33/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0183 - acc: 0.9976 - metric_F1score: 0.9976 - val_loss: 1.4687e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00033: val_acc did not improve from 1.00000 Epoch 34/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0161 - acc: 0.9979 - metric_F1score: 0.9979 - val_loss: 7.5226e-04 - val_acc: 1.0000 - val_metric_F1score: 0.9995 Epoch 00034: val_acc did not improve from 1.00000 Epoch 35/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0376 - acc: 0.9966 - metric_F1score: 0.9966 - val_loss: 4.1024e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00035: val_acc did not improve from 1.00000 Epoch 36/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0172 - acc: 0.9959 - metric_F1score: 0.9962 - val_loss: 9.3964e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00036: val_acc did not improve from 1.00000 Epoch 37/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0282 - acc: 0.9962 - metric_F1score: 0.9962 - val_loss: 8.5974e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00037: val_acc did not improve from 1.00000 Epoch 38/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0136 - acc: 0.9993 - metric_F1score: 0.9993 - val_loss: 7.9662e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00038: val_acc did not improve from 1.00000 Epoch 39/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0503 - acc: 0.9904 - metric_F1score: 0.9907 - val_loss: 0.0077 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00039: val_acc did not improve from 1.00000 Epoch 40/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0667 - acc: 0.9876 - metric_F1score: 0.9875 - val_loss: 0.0535 - val_acc: 0.9850 - val_metric_F1score: 0.9856 Epoch 00040: val_acc did not improve from 1.00000 Epoch 41/50 94/94 [==============================] - 0s 5ms/step - loss: 0.9856 - acc: 0.9574 - metric_F1score: 0.9569 - val_loss: 0.0551 - val_acc: 0.9820 - val_metric_F1score: 0.9819 Epoch 00041: val_acc did not improve from 1.00000 Epoch 42/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1180 - acc: 0.9791 - metric_F1score: 0.9793 - val_loss: 0.0045 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00042: val_acc did not improve from 1.00000 Epoch 43/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0458 - acc: 0.9817 - metric_F1score: 0.9827 - val_loss: 5.3599e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00043: val_acc did not improve from 1.00000 Epoch 44/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0263 - acc: 0.9932 - metric_F1score: 0.9933 - val_loss: 6.2635e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00044: val_acc did not improve from 1.00000 Epoch 45/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0237 - acc: 0.9955 - metric_F1score: 0.9955 - val_loss: 7.3758e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00045: val_acc did not improve from 1.00000 Epoch 46/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0298 - acc: 0.9944 - metric_F1score: 0.9945 - val_loss: 6.0133e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00046: val_acc did not improve from 1.00000 Epoch 47/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0156 - acc: 0.9953 - metric_F1score: 0.9955 - val_loss: 1.6694e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00047: val_acc did not improve from 1.00000 Epoch 48/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0184 - acc: 0.9955 - metric_F1score: 0.9958 - val_loss: 1.3091e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00048: val_acc did not improve from 1.00000 Epoch 49/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0356 - acc: 0.9936 - metric_F1score: 0.9937 - val_loss: 5.0724e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00049: val_acc did not improve from 1.00000 Epoch 50/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0217 - acc: 0.9957 - metric_F1score: 0.9958 - val_loss: 4.0965e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00050: val_acc did not improve from 1.00000
In [ ]:
import matplotlib.pyplot as plt
fig, loss_ax = plt.subplots(figsize=(16, 10))
acc_ax = loss_ax.twinx()
loss_ax.set_xlabel('epoch')
loss_ax.set_ylabel('loss')
acc_ax.plot(history.history['metric_F1score'], 'b', label='train f1')
acc_ax.plot(history.history['val_metric_F1score'], 'g', label='val f1')
acc_ax.set_ylabel('f1-score')
acc_ax.legend(loc='upper left')
plt.show()
훨씬 요동치는 사실을 알게 되었습니다.
5-2) Batch Normalization¶
Batch Normalization은 배치 정규화라고 합니다. 이는 2015년에 제안된 방법인데 효과가 입증되어 많이 쓰이는 기법입니다. 배치 정규화는 과적합을 방지하고 초기값에 크게 의존하지 않으며 학습 속도가 개선되는 장점도 가지고 있습니다.
tf.keras.layers.BatchNormalization( axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True, beta_initializer="zeros", gamma_initializer="ones", moving_mean_initializer="zeros", moving_variance_initializer="ones", beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None, kwargs )**
기본 값만 해도 엄청 많습니다.
In [ ]:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import BatchNormalization
model3 = Sequential([
LSTM(64, activation='relu', input_shape=x_train.shape[1:3]),
Dropout(0.2),
BatchNormalization(),
Dense(32, activation='relu'),
Dropout(0.2),
BatchNormalization(),
Dense(len(actions), activation='softmax')
])
model3.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc', metric_F1score])
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
history3 = model3.fit(
x_train,
y_train,
validation_data=(x_val, y_val),
epochs=50,
callbacks=[
ModelCheckpoint('models/multi_hand_gesture_classifier.h5', monitor='val_acc', verbose=1, save_best_only=True, mode='auto'),
ReduceLROnPlateau(monitor='val_metric_F1score', factor=0.5, patience=50, verbose=1, mode='auto')
]
)
Epoch 1/50 94/94 [==============================] - 3s 8ms/step - loss: 1.0378 - acc: 0.6027 - metric_F1score: 0.5568 - val_loss: 0.3039 - val_acc: 0.9058 - val_metric_F1score: 0.9041 Epoch 00001: val_acc improved from -inf to 0.90581, saving model to models\multi_hand_gesture_classifier.h5 Epoch 2/50 94/94 [==============================] - 0s 5ms/step - loss: 0.3998 - acc: 0.8692 - metric_F1score: 0.8573 - val_loss: 0.2878 - val_acc: 0.9259 - val_metric_F1score: 0.9230 Epoch 00002: val_acc improved from 0.90581 to 0.92585, saving model to models\multi_hand_gesture_classifier.h5 Epoch 3/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2211 - acc: 0.9370 - metric_F1score: 0.9329 - val_loss: 0.0935 - val_acc: 0.9870 - val_metric_F1score: 0.9852 Epoch 00003: val_acc improved from 0.92585 to 0.98697, saving model to models\multi_hand_gesture_classifier.h5 Epoch 4/50 94/94 [==============================] - 0s 5ms/step - loss: 0.1374 - acc: 0.9618 - metric_F1score: 0.9628 - val_loss: 0.0253 - val_acc: 0.9980 - val_metric_F1score: 0.9966 Epoch 00004: val_acc improved from 0.98697 to 0.99800, saving model to models\multi_hand_gesture_classifier.h5 Epoch 5/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0714 - acc: 0.9820 - metric_F1score: 0.9812 - val_loss: 0.0124 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00005: val_acc improved from 0.99800 to 0.99900, saving model to models\multi_hand_gesture_classifier.h5 Epoch 6/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0540 - acc: 0.9890 - metric_F1score: 0.9890 - val_loss: 0.0976 - val_acc: 0.9920 - val_metric_F1score: 0.9927 Epoch 00006: val_acc did not improve from 0.99900 Epoch 7/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0518 - acc: 0.9877 - metric_F1score: 0.9882 - val_loss: 0.0273 - val_acc: 0.9910 - val_metric_F1score: 0.9911 Epoch 00007: val_acc did not improve from 0.99900 Epoch 8/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0482 - acc: 0.9878 - metric_F1score: 0.9878 - val_loss: 0.0363 - val_acc: 0.9840 - val_metric_F1score: 0.9853 Epoch 00008: val_acc did not improve from 0.99900 Epoch 9/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0418 - acc: 0.9911 - metric_F1score: 0.9892 - val_loss: 0.5164 - val_acc: 0.8607 - val_metric_F1score: 0.8641 Epoch 00009: val_acc did not improve from 0.99900 Epoch 10/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1583 - acc: 0.9496 - metric_F1score: 0.9510 - val_loss: 0.0572 - val_acc: 0.9840 - val_metric_F1score: 0.9838 Epoch 00010: val_acc did not improve from 0.99900 Epoch 11/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0494 - acc: 0.9837 - metric_F1score: 0.9846 - val_loss: 0.0124 - val_acc: 0.9960 - val_metric_F1score: 0.9956 Epoch 00011: val_acc did not improve from 0.99900 Epoch 12/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0267 - acc: 0.9938 - metric_F1score: 0.9939 - val_loss: 0.0099 - val_acc: 0.9950 - val_metric_F1score: 0.9970 Epoch 00012: val_acc did not improve from 0.99900 Epoch 13/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0191 - acc: 0.9968 - metric_F1score: 0.9961 - val_loss: 0.0018 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00013: val_acc improved from 0.99900 to 1.00000, saving model to models\multi_hand_gesture_classifier.h5 Epoch 14/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0211 - acc: 0.9944 - metric_F1score: 0.9947 - val_loss: 7.7766e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00014: val_acc did not improve from 1.00000 Epoch 15/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0124 - acc: 0.9979 - metric_F1score: 0.9982 - val_loss: 4.8801e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00015: val_acc did not improve from 1.00000 Epoch 16/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0107 - acc: 0.9975 - metric_F1score: 0.9976 - val_loss: 3.7130e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00016: val_acc did not improve from 1.00000 Epoch 17/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0085 - acc: 0.9991 - metric_F1score: 0.9988 - val_loss: 3.2222e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00017: val_acc did not improve from 1.00000 Epoch 18/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0065 - acc: 0.9997 - metric_F1score: 0.9995 - val_loss: 1.8883e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00018: val_acc did not improve from 1.00000 Epoch 19/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0079 - acc: 0.9988 - metric_F1score: 0.9987 - val_loss: 1.4943e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00019: val_acc did not improve from 1.00000 Epoch 20/50 94/94 [==============================] - 1s 5ms/step - loss: 0.0063 - acc: 0.9994 - metric_F1score: 0.9993 - val_loss: 1.6901e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00020: val_acc did not improve from 1.00000 Epoch 21/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0053 - acc: 0.9994 - metric_F1score: 0.9994 - val_loss: 8.4226e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00021: val_acc did not improve from 1.00000 Epoch 22/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0060 - acc: 0.9989 - metric_F1score: 0.9989 - val_loss: 8.3034e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00022: val_acc did not improve from 1.00000 Epoch 23/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0048 - acc: 0.9997 - metric_F1score: 0.9996 - val_loss: 6.6226e-05 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00023: val_acc did not improve from 1.00000 Epoch 24/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0034 - acc: 0.9997 - metric_F1score: 0.9997 - val_loss: 0.0047 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00024: val_acc did not improve from 1.00000 Epoch 25/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0137 - acc: 0.9971 - metric_F1score: 0.9971 - val_loss: 1.0687e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00025: val_acc did not improve from 1.00000 Epoch 26/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0538 - acc: 0.9843 - metric_F1score: 0.9845 - val_loss: 0.0012 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00026: val_acc did not improve from 1.00000 Epoch 27/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0157 - acc: 0.9972 - metric_F1score: 0.9971 - val_loss: 2.5849e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00027: val_acc did not improve from 1.00000 Epoch 28/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0079 - acc: 0.9986 - metric_F1score: 0.9985 - val_loss: 0.1289 - val_acc: 0.9900 - val_metric_F1score: 0.9902 Epoch 00028: val_acc did not improve from 1.00000 Epoch 29/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0281 - acc: 0.9924 - metric_F1score: 0.9930 - val_loss: 4.6822e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00029: val_acc did not improve from 1.00000 Epoch 30/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0048 - acc: 0.9991 - metric_F1score: 0.9991 - val_loss: 5.8211e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00030: val_acc did not improve from 1.00000 Epoch 31/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0154 - acc: 0.9951 - metric_F1score: 0.9952 - val_loss: 9.3420 - val_acc: 0.2014 - val_metric_F1score: 0.2005 Epoch 00031: val_acc did not improve from 1.00000 Epoch 32/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5049 - acc: 0.8479 - metric_F1score: 0.8510 - val_loss: 0.2898 - val_acc: 0.8798 - val_metric_F1score: 0.8831 Epoch 00032: val_acc did not improve from 1.00000 Epoch 33/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1572 - acc: 0.9485 - metric_F1score: 0.9485 - val_loss: 0.0949 - val_acc: 0.9689 - val_metric_F1score: 0.9675 Epoch 00033: val_acc did not improve from 1.00000 Epoch 34/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0670 - acc: 0.9790 - metric_F1score: 0.9787 - val_loss: 0.0184 - val_acc: 0.9950 - val_metric_F1score: 0.9951 Epoch 00034: val_acc did not improve from 1.00000 Epoch 35/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0598 - acc: 0.9783 - metric_F1score: 0.9790 - val_loss: 0.0125 - val_acc: 0.9980 - val_metric_F1score: 0.9975 Epoch 00035: val_acc did not improve from 1.00000 Epoch 36/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0492 - acc: 0.9821 - metric_F1score: 0.9831 - val_loss: 0.0062 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00036: val_acc did not improve from 1.00000 Epoch 37/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0703 - acc: 0.9774 - metric_F1score: 0.9774 - val_loss: 0.0053 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00037: val_acc did not improve from 1.00000 Epoch 38/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0583 - acc: 0.9790 - metric_F1score: 0.9796 - val_loss: 0.0075 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00038: val_acc did not improve from 1.00000 Epoch 39/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0562 - acc: 0.9851 - metric_F1score: 0.9852 - val_loss: 0.0016 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00039: val_acc did not improve from 1.00000 Epoch 40/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0299 - acc: 0.9912 - metric_F1score: 0.9906 - val_loss: 0.0019 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00040: val_acc did not improve from 1.00000 Epoch 41/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0309 - acc: 0.9909 - metric_F1score: 0.9909 - val_loss: 7.3822e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00041: val_acc did not improve from 1.00000 Epoch 42/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0194 - acc: 0.9941 - metric_F1score: 0.9940 - val_loss: 7.3138e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00042: val_acc did not improve from 1.00000 Epoch 43/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0176 - acc: 0.9955 - metric_F1score: 0.9949 - val_loss: 0.0012 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00043: val_acc did not improve from 1.00000 Epoch 44/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0136 - acc: 0.9969 - metric_F1score: 0.9973 - val_loss: 7.0841e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00044: val_acc did not improve from 1.00000 Epoch 45/50 94/94 [==============================] - 0s 4ms/step - loss: 0.0242 - acc: 0.9941 - metric_F1score: 0.9940 - val_loss: 3.7355e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00045: val_acc did not improve from 1.00000 Epoch 46/50 94/94 [==============================] - 1s 6ms/step - loss: 0.0170 - acc: 0.9961 - metric_F1score: 0.9963 - val_loss: 0.0046 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00046: val_acc did not improve from 1.00000 Epoch 47/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0141 - acc: 0.9949 - metric_F1score: 0.9953 - val_loss: 0.0013 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00047: val_acc did not improve from 1.00000 Epoch 48/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0257 - acc: 0.9937 - metric_F1score: 0.9937 - val_loss: 4.5743e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00048: val_acc did not improve from 1.00000 Epoch 49/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0216 - acc: 0.9937 - metric_F1score: 0.9942 - val_loss: 0.0012 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00049: val_acc did not improve from 1.00000 Epoch 50/50 94/94 [==============================] - 0s 5ms/step - loss: 0.0138 - acc: 0.9961 - metric_F1score: 0.9963 - val_loss: 1.1180e-04 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00050: val_acc did not improve from 1.00000
In [ ]:
import matplotlib.pyplot as plt
fig, loss_ax = plt.subplots(figsize=(16, 10))
acc_ax = loss_ax.twinx()
loss_ax.set_xlabel('epoch')
loss_ax.set_ylabel('loss')
acc_ax.plot(history3.history['metric_F1score'], 'b', label='train f1')
acc_ax.plot(history3.history['val_metric_F1score'], 'g', label='val f1')
acc_ax.set_ylabel('f1-score')
acc_ax.legend(loc='upper left')
plt.show()
5-3) Early Stopping¶
- 여러 과적합을 해결하기 위한 방법을 적용시키고 있는데 바로 위 그래프만 살펴보더라도 특정 숫자(1.0) 근처로 값들의 변동이 있음을 확인할 수 있습니다. 여기 이상으로 학습이 안된다는 의미로 해석할 수 있습니다.
- 이때, 학습이 안되는 시점에서 빠르게 학습을 종료하는 Early Stopping 기법을 이용하여 과하게 학습됨을 방지하겠습니다.
- tensorflow.keras.callbacks.EarlyStopping(monitor, min_deltam patience, verbose, mode, baseline, restore_best_weights)
- monitor : EarlyStopping의 기준이 되는 값
- min_delta : 개선된 것으로 간주하기 위한 최소한의 변화량
- patience : monitor가 개선되지 않을 때 몇 번의 epoch를 진행할 지 정하는 값
- verbose : 0(화면 나타냄x) or 1(화면 나타냄o)
- mode : monitor의 값이 어떤 mode가 되어야 하는지 알려주는 값
- auto
- min
- max
- baseline : 모형이 달성해야 하는 최소한의 기준값, patience 이내에서 모형이 baseline 기준으로 개선됨이 보이지 않으면 중단시킨다.
- restore_best_weights : True(가장 좋을때의 training weight) or False(마지막 training weights)
In [ ]:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import BatchNormalization
model4 = Sequential([
LSTM(64, activation='relu', input_shape=x_train.shape[1:3]),
Dropout(0.5),
BatchNormalization(),
Dense(32, activation='relu'),
Dropout(0.5),
BatchNormalization(),
Dense(len(actions), activation='softmax')
])
model4.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc', metric_F1score])
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
from keras.callbacks import EarlyStopping
early_stopping = EarlyStopping(monitor = 'val_metric_F1score', min_delta = 0, patience = 20, mode = 'auto')
history4 = model4.fit(
x_train,
y_train,
validation_data=(x_val, y_val),
epochs=50,
callbacks=[
ModelCheckpoint('models/multi_hand_gesture_classifier.h5', monitor='val_acc', verbose=1, save_best_only=True, mode='auto'),
ReduceLROnPlateau(monitor='val_metric_F1score', factor=0.5, patience=50, verbose=1, mode='auto'),
early_stopping
]
)
Epoch 1/50 94/94 [==============================] - 3s 13ms/step - loss: 1.4969 - acc: 0.3849 - metric_F1score: 0.3043 - val_loss: 0.6401 - val_acc: 0.8527 - val_metric_F1score: 0.6680 Epoch 00001: val_acc improved from -inf to 0.85271, saving model to models\multi_hand_gesture_classifier.h5 Epoch 2/50 94/94 [==============================] - 1s 5ms/step - loss: 0.8276 - acc: 0.6607 - metric_F1score: 0.6239 - val_loss: 0.3299 - val_acc: 0.9319 - val_metric_F1score: 0.9170 Epoch 00002: val_acc improved from 0.85271 to 0.93186, saving model to models\multi_hand_gesture_classifier.h5 Epoch 3/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5768 - acc: 0.7944 - metric_F1score: 0.7660 - val_loss: 0.2087 - val_acc: 0.9479 - val_metric_F1score: 0.9483 Epoch 00003: val_acc improved from 0.93186 to 0.94790, saving model to models\multi_hand_gesture_classifier.h5 Epoch 4/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4423 - acc: 0.8692 - metric_F1score: 0.8527 - val_loss: 0.2618 - val_acc: 0.9088 - val_metric_F1score: 0.9074 Epoch 00004: val_acc did not improve from 0.94790 Epoch 5/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4018 - acc: 0.8624 - metric_F1score: 0.8538 - val_loss: 0.0865 - val_acc: 0.9900 - val_metric_F1score: 0.9907 Epoch 00005: val_acc improved from 0.94790 to 0.98998, saving model to models\multi_hand_gesture_classifier.h5 Epoch 6/50 94/94 [==============================] - 0s 5ms/step - loss: 0.2540 - acc: 0.9327 - metric_F1score: 0.9328 - val_loss: 0.0567 - val_acc: 0.9930 - val_metric_F1score: 0.9927 Epoch 00006: val_acc improved from 0.98998 to 0.99299, saving model to models\multi_hand_gesture_classifier.h5 Epoch 7/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1885 - acc: 0.9516 - metric_F1score: 0.9500 - val_loss: 0.0209 - val_acc: 0.9970 - val_metric_F1score: 0.9971 Epoch 00007: val_acc improved from 0.99299 to 0.99699, saving model to models\multi_hand_gesture_classifier.h5 Epoch 8/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1472 - acc: 0.9682 - metric_F1score: 0.9634 - val_loss: 0.0570 - val_acc: 0.9940 - val_metric_F1score: 0.9936 Epoch 00008: val_acc did not improve from 0.99699 Epoch 9/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1273 - acc: 0.9693 - metric_F1score: 0.9671 - val_loss: 0.0135 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00009: val_acc improved from 0.99699 to 0.99800, saving model to models\multi_hand_gesture_classifier.h5 Epoch 10/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1075 - acc: 0.9772 - metric_F1score: 0.9737 - val_loss: 0.0114 - val_acc: 0.9990 - val_metric_F1score: 0.9985 Epoch 00010: val_acc improved from 0.99800 to 0.99900, saving model to models\multi_hand_gesture_classifier.h5 Epoch 11/50 94/94 [==============================] - 0s 5ms/step - loss: 0.1201 - acc: 0.9709 - metric_F1score: 0.9695 - val_loss: 0.0214 - val_acc: 0.9950 - val_metric_F1score: 0.9956 Epoch 00011: val_acc did not improve from 0.99900 Epoch 12/50 94/94 [==============================] - 1s 5ms/step - loss: 0.1149 - acc: 0.9686 - metric_F1score: 0.9677 - val_loss: 0.0223 - val_acc: 0.9940 - val_metric_F1score: 0.9941 Epoch 00012: val_acc did not improve from 0.99900 Epoch 13/50 94/94 [==============================] - 1s 5ms/step - loss: 0.1167 - acc: 0.9708 - metric_F1score: 0.9723 - val_loss: 0.0457 - val_acc: 0.9900 - val_metric_F1score: 0.9907 Epoch 00013: val_acc did not improve from 0.99900 Epoch 14/50 94/94 [==============================] - 0s 4ms/step - loss: 0.1095 - acc: 0.9696 - metric_F1score: 0.9694 - val_loss: 0.2856 - val_acc: 0.9529 - val_metric_F1score: 0.9524 Epoch 00014: val_acc did not improve from 0.99900 Epoch 15/50 94/94 [==============================] - 0s 5ms/step - loss: 0.2307 - acc: 0.9323 - metric_F1score: 0.9342 - val_loss: 3.3327 - val_acc: 0.7836 - val_metric_F1score: 0.7820 Epoch 00015: val_acc did not improve from 0.99900 Epoch 16/50 94/94 [==============================] - 0s 5ms/step - loss: 0.5387 - acc: 0.8223 - metric_F1score: 0.8197 - val_loss: 0.1757 - val_acc: 0.9529 - val_metric_F1score: 0.9499 Epoch 00016: val_acc did not improve from 0.99900 Epoch 17/50 94/94 [==============================] - 0s 5ms/step - loss: 0.2380 - acc: 0.9212 - metric_F1score: 0.9165 - val_loss: 1.5321 - val_acc: 0.9830 - val_metric_F1score: 0.9812 Epoch 00017: val_acc did not improve from 0.99900 Epoch 18/50 94/94 [==============================] - 1s 6ms/step - loss: 0.1962 - acc: 0.9329 - metric_F1score: 0.9338 - val_loss: 0.0519 - val_acc: 0.9870 - val_metric_F1score: 0.9882 Epoch 00018: val_acc did not improve from 0.99900 Epoch 19/50 94/94 [==============================] - 1s 7ms/step - loss: 0.1802 - acc: 0.9497 - metric_F1score: 0.9465 - val_loss: 0.0570 - val_acc: 0.9850 - val_metric_F1score: 0.9843 Epoch 00019: val_acc did not improve from 0.99900 Epoch 20/50 94/94 [==============================] - 1s 6ms/step - loss: 0.1467 - acc: 0.9562 - metric_F1score: 0.9565 - val_loss: 0.0526 - val_acc: 0.9850 - val_metric_F1score: 0.9853 Epoch 00020: val_acc did not improve from 0.99900 Epoch 21/50 94/94 [==============================] - 1s 8ms/step - loss: 0.1426 - acc: 0.9609 - metric_F1score: 0.9617 - val_loss: 0.0198 - val_acc: 0.9960 - val_metric_F1score: 0.9951 Epoch 00021: val_acc did not improve from 0.99900
In [ ]:
import matplotlib.pyplot as plt
fig, loss_ax = plt.subplots(figsize=(16, 10))
acc_ax = loss_ax.twinx()
loss_ax.set_xlabel('epoch')
loss_ax.set_ylabel('loss')
acc_ax.plot(history4.history['metric_F1score'], 'b', label='train f1')
acc_ax.plot(history4.history['val_metric_F1score'], 'g', label='val f1')
acc_ax.set_ylabel('f1-score')
acc_ax.legend(loc='upper left')
plt.show()
5-4) 규제 방법 적용¶
Regularization(정규화, 규제, 일반화) 이란 모델이 과적합이 되도록 학습하지 않고 일반성을 가질 수 있도록 가중치(weight) 를 조정하여 규제하는 것을 의미합니다.
대표적으로 L1 규제(Lasso)와 L2 규제(Lidge)가 있습니다.
L1 규제에서 가장 중요한 것은 손실 함수(loss function)에 가중치의 절대값을 더해 준다는 것입니다. L1 규제는 아래의 수식처럼 표현할 수 있습니다.
L1 규제의 경우에는 가중치의 크기에 상관없이 상수값을 뺍니다. 이는 대체적으로 불필요한 가중치의 수치를 0으로 만들도록 하는 방향으로 적용됩니다. 즉, 중요한 가중치만을 취하기 때문에 희소 데이터(sparse feature)에 대한 모델을 구성하는데 적합합니다.
In [ ]:
#기본 모델 + l1 규제
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout
import keras.layers
model5 = Sequential([
LSTM(64, activation='relu', input_shape=x_train.shape[1:3], kernel_regularizer=keras.regularizers.l1(0.01)),
Dense(32, activation='relu',kernel_regularizer=keras.regularizers.l1(0.01)),
Dense(len(actions), activation='softmax', kernel_regularizer=keras.regularizers.l1(0.01))
])
model5.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc', metric_F1score])
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
history5 = model5.fit(
x_train,
y_train,
validation_data=(x_val, y_val),
epochs=50,
callbacks=[
ModelCheckpoint('models/multi_hand_gesture_classifier.h5', monitor='val_acc', verbose=1, save_best_only=True, mode='auto'),
ReduceLROnPlateau(monitor='val_metric_F1score', factor=0.5, patience=50, verbose=1, mode='auto')
]
)
Epoch 1/50 94/94 [==============================] - 2s 8ms/step - loss: 24.8018 - acc: 0.6969 - metric_F1score: 0.6971 - val_loss: 12.1133 - val_acc: 0.9248 - val_metric_F1score: 0.9178 Epoch 00001: val_acc improved from -inf to 0.92485, saving model to models\multi_hand_gesture_classifier.h5 Epoch 2/50 94/94 [==============================] - 0s 5ms/step - loss: 10.9349 - acc: 0.9323 - metric_F1score: 0.9325 - val_loss: 8.7661 - val_acc: 0.9449 - val_metric_F1score: 0.9463 Epoch 00002: val_acc improved from 0.92485 to 0.94489, saving model to models\multi_hand_gesture_classifier.h5 Epoch 3/50 94/94 [==============================] - 0s 4ms/step - loss: 8.3471 - acc: 0.9466 - metric_F1score: 0.9466 - val_loss: 7.2125 - val_acc: 0.9729 - val_metric_F1score: 0.9741 Epoch 00003: val_acc improved from 0.94489 to 0.97295, saving model to models\multi_hand_gesture_classifier.h5 Epoch 4/50 94/94 [==============================] - 0s 4ms/step - loss: 6.8617 - acc: 0.9872 - metric_F1score: 0.9871 - val_loss: 6.2328 - val_acc: 0.9880 - val_metric_F1score: 0.9887 Epoch 00004: val_acc improved from 0.97295 to 0.98798, saving model to models\multi_hand_gesture_classifier.h5 Epoch 5/50 94/94 [==============================] - 0s 4ms/step - loss: 6.0103 - acc: 0.9888 - metric_F1score: 0.9885 - val_loss: 5.5751 - val_acc: 0.9850 - val_metric_F1score: 0.9811 Epoch 00005: val_acc did not improve from 0.98798 Epoch 6/50 94/94 [==============================] - 0s 4ms/step - loss: 5.4424 - acc: 0.9852 - metric_F1score: 0.9851 - val_loss: 5.1420 - val_acc: 0.9860 - val_metric_F1score: 0.9858 Epoch 00006: val_acc did not improve from 0.98798 Epoch 7/50 94/94 [==============================] - 0s 4ms/step - loss: 4.9904 - acc: 0.9914 - metric_F1score: 0.9920 - val_loss: 4.6965 - val_acc: 0.9860 - val_metric_F1score: 0.9863 Epoch 00007: val_acc did not improve from 0.98798 Epoch 8/50 94/94 [==============================] - 0s 4ms/step - loss: 4.5584 - acc: 0.9968 - metric_F1score: 0.9968 - val_loss: 4.3570 - val_acc: 0.9870 - val_metric_F1score: 0.9873 Epoch 00008: val_acc did not improve from 0.98798 Epoch 9/50 94/94 [==============================] - 0s 4ms/step - loss: 4.3346 - acc: 0.9925 - metric_F1score: 0.9925 - val_loss: 4.8223 - val_acc: 0.9499 - val_metric_F1score: 0.9512 Epoch 00009: val_acc did not improve from 0.98798 Epoch 10/50 94/94 [==============================] - 0s 4ms/step - loss: 4.4272 - acc: 0.9657 - metric_F1score: 0.9658 - val_loss: 4.2358 - val_acc: 0.9569 - val_metric_F1score: 0.9527 Epoch 00010: val_acc did not improve from 0.98798 Epoch 11/50 94/94 [==============================] - 0s 4ms/step - loss: 4.2076 - acc: 0.9724 - metric_F1score: 0.9732 - val_loss: 4.0022 - val_acc: 0.9800 - val_metric_F1score: 0.9762 Epoch 00011: val_acc did not improve from 0.98798 Epoch 12/50 94/94 [==============================] - 0s 4ms/step - loss: 3.8005 - acc: 0.9905 - metric_F1score: 0.9906 - val_loss: 3.5857 - val_acc: 0.9950 - val_metric_F1score: 0.9951 Epoch 00012: val_acc improved from 0.98798 to 0.99499, saving model to models\multi_hand_gesture_classifier.h5 Epoch 13/50 94/94 [==============================] - 0s 4ms/step - loss: 3.5340 - acc: 0.9981 - metric_F1score: 0.9981 - val_loss: 3.4116 - val_acc: 0.9960 - val_metric_F1score: 0.9966 Epoch 00013: val_acc improved from 0.99499 to 0.99599, saving model to models\multi_hand_gesture_classifier.h5 Epoch 14/50 94/94 [==============================] - 0s 4ms/step - loss: 3.3637 - acc: 0.9981 - metric_F1score: 0.9980 - val_loss: 3.2598 - val_acc: 0.9950 - val_metric_F1score: 0.9956 Epoch 00014: val_acc did not improve from 0.99599 Epoch 15/50 94/94 [==============================] - 0s 4ms/step - loss: 3.2195 - acc: 0.9990 - metric_F1score: 0.9990 - val_loss: 3.1195 - val_acc: 0.9960 - val_metric_F1score: 0.9956 Epoch 00015: val_acc did not improve from 0.99599 Epoch 16/50 94/94 [==============================] - 0s 4ms/step - loss: 3.0820 - acc: 0.9986 - metric_F1score: 0.9986 - val_loss: 2.9894 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00016: val_acc improved from 0.99599 to 0.99800, saving model to models\multi_hand_gesture_classifier.h5 Epoch 17/50 94/94 [==============================] - 0s 4ms/step - loss: 2.9753 - acc: 0.9944 - metric_F1score: 0.9944 - val_loss: 2.8801 - val_acc: 0.9980 - val_metric_F1score: 0.9976 Epoch 00017: val_acc did not improve from 0.99800 Epoch 18/50 94/94 [==============================] - 0s 5ms/step - loss: 2.8507 - acc: 0.9982 - metric_F1score: 0.9982 - val_loss: 2.7744 - val_acc: 0.9970 - val_metric_F1score: 0.9971 Epoch 00018: val_acc did not improve from 0.99800 Epoch 19/50 94/94 [==============================] - 0s 4ms/step - loss: 2.7388 - acc: 0.9998 - metric_F1score: 0.9998 - val_loss: 2.6676 - val_acc: 0.9970 - val_metric_F1score: 0.9971 Epoch 00019: val_acc did not improve from 0.99800 Epoch 20/50 94/94 [==============================] - 0s 4ms/step - loss: 2.6361 - acc: 0.9993 - metric_F1score: 0.9993 - val_loss: 2.5621 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00020: val_acc did not improve from 0.99800 Epoch 21/50 94/94 [==============================] - 0s 4ms/step - loss: 2.5350 - acc: 0.9996 - metric_F1score: 0.9996 - val_loss: 2.4642 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00021: val_acc improved from 0.99800 to 1.00000, saving model to models\multi_hand_gesture_classifier.h5 Epoch 22/50 94/94 [==============================] - 0s 5ms/step - loss: 2.4419 - acc: 0.9990 - metric_F1score: 0.9990 - val_loss: 2.3847 - val_acc: 0.9980 - val_metric_F1score: 0.9985 Epoch 00022: val_acc did not improve from 1.00000 Epoch 23/50 94/94 [==============================] - 0s 4ms/step - loss: 2.4019 - acc: 0.9923 - metric_F1score: 0.9926 - val_loss: 2.5806 - val_acc: 0.9549 - val_metric_F1score: 0.9476 Epoch 00023: val_acc did not improve from 1.00000 Epoch 24/50 94/94 [==============================] - 0s 4ms/step - loss: 2.4608 - acc: 0.9835 - metric_F1score: 0.9824 - val_loss: 2.2655 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00024: val_acc did not improve from 1.00000 Epoch 25/50 94/94 [==============================] - 0s 4ms/step - loss: 2.2347 - acc: 0.9994 - metric_F1score: 0.9992 - val_loss: 2.1605 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00025: val_acc did not improve from 1.00000 Epoch 26/50 94/94 [==============================] - 0s 4ms/step - loss: 2.1337 - acc: 0.9994 - metric_F1score: 0.9996 - val_loss: 2.0676 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00026: val_acc did not improve from 1.00000 Epoch 27/50 94/94 [==============================] - 0s 4ms/step - loss: 2.0480 - acc: 0.9994 - metric_F1score: 0.9997 - val_loss: 1.9883 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00027: val_acc did not improve from 1.00000 Epoch 28/50 94/94 [==============================] - 0s 4ms/step - loss: 1.9686 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.9139 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00028: val_acc did not improve from 1.00000 Epoch 29/50 94/94 [==============================] - 0s 4ms/step - loss: 1.9036 - acc: 0.9989 - metric_F1score: 0.9988 - val_loss: 2.0478 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00029: val_acc did not improve from 1.00000 Epoch 30/50 94/94 [==============================] - 0s 4ms/step - loss: 2.0044 - acc: 0.9989 - metric_F1score: 0.9989 - val_loss: 1.8692 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00030: val_acc did not improve from 1.00000 Epoch 31/50 94/94 [==============================] - 0s 4ms/step - loss: 1.8409 - acc: 0.9994 - metric_F1score: 0.9994 - val_loss: 1.7690 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00031: val_acc did not improve from 1.00000 Epoch 32/50 94/94 [==============================] - 0s 4ms/step - loss: 1.7442 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.6839 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00032: val_acc did not improve from 1.00000 Epoch 33/50 94/94 [==============================] - 0s 4ms/step - loss: 1.6635 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.6137 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00033: val_acc did not improve from 1.00000 Epoch 34/50 94/94 [==============================] - 0s 4ms/step - loss: 1.5927 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.5474 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00034: val_acc did not improve from 1.00000 Epoch 35/50 94/94 [==============================] - 0s 4ms/step - loss: 1.5292 - acc: 0.9999 - metric_F1score: 0.9999 - val_loss: 1.4905 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00035: val_acc did not improve from 1.00000 Epoch 36/50 94/94 [==============================] - 0s 4ms/step - loss: 1.4717 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.4263 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00036: val_acc did not improve from 1.00000 Epoch 37/50 94/94 [==============================] - 0s 4ms/step - loss: 1.4163 - acc: 0.9993 - metric_F1score: 0.9993 - val_loss: 1.3984 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00037: val_acc did not improve from 1.00000 Epoch 38/50 94/94 [==============================] - 0s 4ms/step - loss: 1.3772 - acc: 0.9996 - metric_F1score: 0.9996 - val_loss: 1.3251 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00038: val_acc did not improve from 1.00000 Epoch 39/50 94/94 [==============================] - 0s 4ms/step - loss: 1.3101 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.2702 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00039: val_acc did not improve from 1.00000 Epoch 40/50 94/94 [==============================] - 0s 4ms/step - loss: 1.2772 - acc: 0.9979 - metric_F1score: 0.9979 - val_loss: 1.3813 - val_acc: 0.9980 - val_metric_F1score: 0.9985 Epoch 00040: val_acc did not improve from 1.00000 Epoch 41/50 94/94 [==============================] - 0s 4ms/step - loss: 1.3313 - acc: 0.9980 - metric_F1score: 0.9982 - val_loss: 1.2260 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00041: val_acc did not improve from 1.00000 Epoch 42/50 94/94 [==============================] - 0s 4ms/step - loss: 1.2058 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.1531 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00042: val_acc did not improve from 1.00000 Epoch 43/50 94/94 [==============================] - 0s 4ms/step - loss: 1.1393 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 1.1015 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00043: val_acc did not improve from 1.00000 Epoch 44/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0878 - acc: 0.9998 - metric_F1score: 0.9998 - val_loss: 1.0498 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00044: val_acc did not improve from 1.00000 Epoch 45/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0922 - acc: 0.9917 - metric_F1score: 0.9919 - val_loss: 1.5590 - val_acc: 0.9770 - val_metric_F1score: 0.9760 Epoch 00045: val_acc did not improve from 1.00000 Epoch 46/50 94/94 [==============================] - 0s 4ms/step - loss: 1.4274 - acc: 0.9959 - metric_F1score: 0.9958 - val_loss: 1.2304 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00046: val_acc did not improve from 1.00000 Epoch 47/50 94/94 [==============================] - 0s 4ms/step - loss: 1.2318 - acc: 0.9940 - metric_F1score: 0.9936 - val_loss: 1.1200 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00047: val_acc did not improve from 1.00000 Epoch 48/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0925 - acc: 0.9996 - metric_F1score: 0.9996 - val_loss: 1.0331 - val_acc: 0.9980 - val_metric_F1score: 0.9985 Epoch 00048: val_acc did not improve from 1.00000 Epoch 49/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0272 - acc: 0.9975 - metric_F1score: 0.9977 - val_loss: 1.1733 - val_acc: 0.9820 - val_metric_F1score: 0.9819 Epoch 00049: val_acc did not improve from 1.00000 Epoch 50/50 94/94 [==============================] - 0s 4ms/step - loss: 1.1544 - acc: 0.9857 - metric_F1score: 0.9850 - val_loss: 1.0224 - val_acc: 0.9980 - val_metric_F1score: 0.9985 Epoch 00050: val_acc did not improve from 1.00000
In [ ]:
import matplotlib.pyplot as plt
fig, loss_ax = plt.subplots(figsize=(16, 10))
acc_ax = loss_ax.twinx()
loss_ax.set_xlabel('epoch')
loss_ax.set_ylabel('loss')
acc_ax.plot(history5.history['metric_F1score'], 'b', label='train f1')
acc_ax.plot(history5.history['val_metric_F1score'], 'g', label='val f1')
acc_ax.set_ylabel('f1-score')
acc_ax.legend(loc='upper left')
plt.show()
L2 규제는 기존의 손실 함수(loss function)에 가중치의 제곱을 더함으로써 L1 규제와 마찬가지로 가중치가 너무 크지 않은 방향으로 학습하도록 합니다. L2 규제는 아래의 수식처럼 표현 할 수 있습니다.
L2의 경우에는 가중치의 값을 이용합니다. 이상치나 노이즈가 있는 데이터에 대한 학습을 진행할 때 사용하면 좋습니다. 특히 선형 모델(Linear model)의 일반화에 좋다고 합니다.
In [ ]:
#기본 모델 + l2 규제
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout
import keras.layers
model5 = Sequential([
LSTM(64, activation='relu', input_shape=x_train.shape[1:3], kernel_regularizer=keras.regularizers.l2(0.01)),
Dense(32, activation='relu',kernel_regularizer=keras.regularizers.l2(0.01)),
Dense(len(actions), activation='softmax', kernel_regularizer=keras.regularizers.l2(0.01))
])
model5.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc', metric_F1score])
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
history5 = model5.fit(
x_train,
y_train,
validation_data=(x_val, y_val),
epochs=50,
callbacks=[
ModelCheckpoint('models/multi_hand_gesture_classifier.h5', monitor='val_acc', verbose=1, save_best_only=True, mode='auto'),
ReduceLROnPlateau(monitor='val_metric_F1score', factor=0.5, patience=50, verbose=1, mode='auto')
]
)
Epoch 1/50 94/94 [==============================] - 2s 11ms/step - loss: 19.9825 - acc: 0.6092 - metric_F1score: 0.6081 - val_loss: 3.0308 - val_acc: 0.8607 - val_metric_F1score: 0.8600 Epoch 00001: val_acc improved from -inf to 0.86072, saving model to models\multi_hand_gesture_classifier.h5 Epoch 2/50 94/94 [==============================] - 0s 4ms/step - loss: 2.8315 - acc: 0.8832 - metric_F1score: 0.8835 - val_loss: 1.3683 - val_acc: 0.9699 - val_metric_F1score: 0.9706 Epoch 00002: val_acc improved from 0.86072 to 0.96994, saving model to models\multi_hand_gesture_classifier.h5 Epoch 3/50 94/94 [==============================] - 0s 4ms/step - loss: 1.2576 - acc: 0.9770 - metric_F1score: 0.9769 - val_loss: 1.1658 - val_acc: 0.9840 - val_metric_F1score: 0.9838 Epoch 00003: val_acc improved from 0.96994 to 0.98397, saving model to models\multi_hand_gesture_classifier.h5 Epoch 4/50 94/94 [==============================] - 0s 4ms/step - loss: 1.1326 - acc: 0.9845 - metric_F1score: 0.9845 - val_loss: 1.0322 - val_acc: 0.9930 - val_metric_F1score: 0.9927 Epoch 00004: val_acc improved from 0.98397 to 0.99299, saving model to models\multi_hand_gesture_classifier.h5 Epoch 5/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0298 - acc: 0.9930 - metric_F1score: 0.9930 - val_loss: 1.0012 - val_acc: 0.9880 - val_metric_F1score: 0.9888 Epoch 00005: val_acc did not improve from 0.99299 Epoch 6/50 94/94 [==============================] - 0s 4ms/step - loss: 0.9526 - acc: 0.9960 - metric_F1score: 0.9960 - val_loss: 0.9216 - val_acc: 0.9900 - val_metric_F1score: 0.9902 Epoch 00006: val_acc did not improve from 0.99299 Epoch 7/50 94/94 [==============================] - 0s 4ms/step - loss: 0.9251 - acc: 0.9924 - metric_F1score: 0.9924 - val_loss: 1.3566 - val_acc: 0.9479 - val_metric_F1score: 0.9492 Epoch 00007: val_acc did not improve from 0.99299 Epoch 8/50 94/94 [==============================] - 0s 4ms/step - loss: 0.9735 - acc: 0.9805 - metric_F1score: 0.9806 - val_loss: 0.8690 - val_acc: 0.9880 - val_metric_F1score: 0.9883 Epoch 00008: val_acc did not improve from 0.99299 Epoch 9/50 94/94 [==============================] - 0s 4ms/step - loss: 0.8369 - acc: 0.9891 - metric_F1score: 0.9891 - val_loss: 0.7810 - val_acc: 0.9970 - val_metric_F1score: 0.9976 Epoch 00009: val_acc improved from 0.99299 to 0.99699, saving model to models\multi_hand_gesture_classifier.h5 Epoch 10/50 94/94 [==============================] - 0s 4ms/step - loss: 0.7767 - acc: 0.9964 - metric_F1score: 0.9964 - val_loss: 0.7406 - val_acc: 0.9980 - val_metric_F1score: 0.9985 Epoch 00010: val_acc improved from 0.99699 to 0.99800, saving model to models\multi_hand_gesture_classifier.h5 Epoch 11/50 94/94 [==============================] - 0s 4ms/step - loss: 0.7308 - acc: 0.9997 - metric_F1score: 0.9998 - val_loss: 0.7068 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00011: val_acc improved from 0.99800 to 0.99900, saving model to models\multi_hand_gesture_classifier.h5 Epoch 12/50 94/94 [==============================] - 0s 4ms/step - loss: 0.6983 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.6762 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00012: val_acc improved from 0.99900 to 1.00000, saving model to models\multi_hand_gesture_classifier.h5 Epoch 13/50 94/94 [==============================] - 0s 4ms/step - loss: 0.6684 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.6486 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00013: val_acc did not improve from 1.00000 Epoch 14/50 94/94 [==============================] - 0s 4ms/step - loss: 0.6721 - acc: 0.9980 - metric_F1score: 0.9980 - val_loss: 0.6381 - val_acc: 0.9950 - val_metric_F1score: 0.9956 Epoch 00014: val_acc did not improve from 1.00000 Epoch 15/50 94/94 [==============================] - 0s 4ms/step - loss: 0.6203 - acc: 0.9998 - metric_F1score: 0.9998 - val_loss: 0.6109 - val_acc: 0.9970 - val_metric_F1score: 0.9971 Epoch 00015: val_acc did not improve from 1.00000 Epoch 16/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5957 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.5821 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00016: val_acc did not improve from 1.00000 Epoch 17/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5734 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.5608 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00017: val_acc did not improve from 1.00000 Epoch 18/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5527 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.5412 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00018: val_acc did not improve from 1.00000 Epoch 19/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5333 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.5225 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00019: val_acc did not improve from 1.00000 Epoch 20/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5150 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.5050 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00020: val_acc did not improve from 1.00000 Epoch 21/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4977 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.4884 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00021: val_acc did not improve from 1.00000 Epoch 22/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4815 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.4726 - val_acc: 0.9980 - val_metric_F1score: 0.9985 Epoch 00022: val_acc did not improve from 1.00000 Epoch 23/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4660 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.4574 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00023: val_acc did not improve from 1.00000 Epoch 24/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4516 - acc: 0.9999 - metric_F1score: 0.9999 - val_loss: 0.4675 - val_acc: 0.9950 - val_metric_F1score: 0.9946 Epoch 00024: val_acc did not improve from 1.00000 Epoch 25/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4459 - acc: 0.9997 - metric_F1score: 0.9997 - val_loss: 0.4328 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00025: val_acc did not improve from 1.00000 Epoch 26/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4268 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.4189 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00026: val_acc did not improve from 1.00000 Epoch 27/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4134 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.4063 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00027: val_acc did not improve from 1.00000 Epoch 28/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4008 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.3942 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00028: val_acc did not improve from 1.00000 Epoch 29/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3889 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.3825 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00029: val_acc did not improve from 1.00000 Epoch 30/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3774 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.3719 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00030: val_acc did not improve from 1.00000 Epoch 31/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3781 - acc: 0.9977 - metric_F1score: 0.9976 - val_loss: 0.3774 - val_acc: 0.9950 - val_metric_F1score: 0.9951 Epoch 00031: val_acc did not improve from 1.00000 Epoch 32/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3704 - acc: 0.9983 - metric_F1score: 0.9983 - val_loss: 0.3536 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00032: val_acc did not improve from 1.00000 Epoch 33/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3507 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.3424 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00033: val_acc did not improve from 1.00000 Epoch 34/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3398 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.3323 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00034: val_acc did not improve from 1.00000 Epoch 35/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3298 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.3296 - val_acc: 0.9970 - val_metric_F1score: 0.9976 Epoch 00035: val_acc did not improve from 1.00000 Epoch 36/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3268 - acc: 0.9979 - metric_F1score: 0.9979 - val_loss: 0.3154 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00036: val_acc did not improve from 1.00000 Epoch 37/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3128 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.3059 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00037: val_acc did not improve from 1.00000 Epoch 38/50 94/94 [==============================] - 0s 4ms/step - loss: 0.3035 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2970 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00038: val_acc did not improve from 1.00000 Epoch 39/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2947 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2886 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00039: val_acc did not improve from 1.00000 Epoch 40/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2863 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2804 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00040: val_acc did not improve from 1.00000 Epoch 41/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2781 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2725 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00041: val_acc did not improve from 1.00000 Epoch 42/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2702 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2648 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00042: val_acc did not improve from 1.00000 Epoch 43/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2624 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2576 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00043: val_acc did not improve from 1.00000 Epoch 44/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2549 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2513 - val_acc: 0.9990 - val_metric_F1score: 0.9990 Epoch 00044: val_acc did not improve from 1.00000 Epoch 45/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2476 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2427 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00045: val_acc did not improve from 1.00000 Epoch 46/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2405 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2358 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00046: val_acc did not improve from 1.00000 Epoch 47/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2335 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2297 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00047: val_acc did not improve from 1.00000 Epoch 48/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2267 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2221 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00048: val_acc did not improve from 1.00000 Epoch 49/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2200 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2158 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00049: val_acc did not improve from 1.00000 Epoch 50/50 94/94 [==============================] - 0s 4ms/step - loss: 0.2140 - acc: 1.0000 - metric_F1score: 1.0000 - val_loss: 0.2146 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00050: val_acc did not improve from 1.00000
In [ ]:
import matplotlib.pyplot as plt
fig, loss_ax = plt.subplots(figsize=(16, 10))
acc_ax = loss_ax.twinx()
loss_ax.set_xlabel('epoch')
loss_ax.set_ylabel('loss')
acc_ax.plot(history5.history['metric_F1score'], 'b', label='train f1')
acc_ax.plot(history5.history['val_metric_F1score'], 'g', label='val f1')
acc_ax.set_ylabel('f1-score')
acc_ax.legend(loc='upper left')
plt.show()
이제 위 규제 방법들을 활용하여 가장 최적으로 데이터를 잘 학습시킨 모형을 찾아보겠습니다.
6. 최종 모형¶
위 데이터에 가장 잘 적합한다고 생각한 최종모형은 LSTM에 L2 규제를 적용하고 30%의 Dropout과 Early Stopping을 적용한 모형입니다. 여러 실험을 해보았는데, 우선 데이터가 쉬워서 무슨 모형을 해도 과적합이 되기 때문에 천천히 1에 도달하는 모형을 select 했습니다.
In [ ]:
#기본 모델 + ㅣ2 규제 + dropout 30% + early stopping
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout
import keras.layers
model7 = Sequential([
LSTM(64, activation='relu', input_shape=x_train.shape[1:3], kernel_regularizer=keras.regularizers.l2(0.01)),
Dropout(0.3),
Dense(32, activation='relu',kernel_regularizer=keras.regularizers.l2(0.01)),
Dropout(0.3),
Dense(len(actions), activation='softmax', kernel_regularizer=keras.regularizers.l2(0.01))
])
model7.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc', metric_F1score])
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
from keras.callbacks import EarlyStopping
early_stopping = EarlyStopping(monitor = 'val_metric_F1score', min_delta = 0, patience = 20, mode = 'auto')
history7 = model7.fit(
x_train,
y_train,
validation_data=(x_val, y_val),
epochs=50,
callbacks=[
ModelCheckpoint('models/multi_hand_gesture_classifier.h5', monitor='val_acc', verbose=1, save_best_only=True, mode='auto'),
ReduceLROnPlateau(monitor='val_metric_F1score', factor=0.5, patience=50, verbose=1, mode='auto'),
early_stopping
]
)
Epoch 1/50 94/94 [==============================] - 2s 7ms/step - loss: 14.1750 - acc: 0.3844 - metric_F1score: 0.3844 - val_loss: 1.7649 - val_acc: 0.9339 - val_metric_F1score: 0.8780 Epoch 00001: val_acc improved from -inf to 0.93387, saving model to models\multi_hand_gesture_classifier.h5 Epoch 2/50 94/94 [==============================] - 0s 4ms/step - loss: 2.3318 - acc: 0.7341 - metric_F1score: 0.7322 - val_loss: 1.2715 - val_acc: 0.9479 - val_metric_F1score: 0.9442 Epoch 00002: val_acc improved from 0.93387 to 0.94790, saving model to models\multi_hand_gesture_classifier.h5 Epoch 3/50 94/94 [==============================] - 0s 4ms/step - loss: 1.5817 - acc: 0.8421 - metric_F1score: 0.8477 - val_loss: 1.0190 - val_acc: 0.9800 - val_metric_F1score: 0.9794 Epoch 00003: val_acc improved from 0.94790 to 0.97996, saving model to models\multi_hand_gesture_classifier.h5 Epoch 4/50 94/94 [==============================] - 0s 5ms/step - loss: 1.2509 - acc: 0.9088 - metric_F1score: 0.8976 - val_loss: 0.8902 - val_acc: 0.9880 - val_metric_F1score: 0.9859 Epoch 00004: val_acc improved from 0.97996 to 0.98798, saving model to models\multi_hand_gesture_classifier.h5 Epoch 5/50 94/94 [==============================] - 0s 4ms/step - loss: 1.0140 - acc: 0.9530 - metric_F1score: 0.9477 - val_loss: 0.7877 - val_acc: 0.9950 - val_metric_F1score: 0.9909 Epoch 00005: val_acc improved from 0.98798 to 0.99499, saving model to models\multi_hand_gesture_classifier.h5 Epoch 6/50 94/94 [==============================] - 0s 4ms/step - loss: 0.8584 - acc: 0.9764 - metric_F1score: 0.9663 - val_loss: 0.7156 - val_acc: 0.9970 - val_metric_F1score: 0.9947 Epoch 00006: val_acc improved from 0.99499 to 0.99699, saving model to models\multi_hand_gesture_classifier.h5 Epoch 7/50 94/94 [==============================] - 0s 4ms/step - loss: 0.8721 - acc: 0.9578 - metric_F1score: 0.9556 - val_loss: 0.6704 - val_acc: 0.9970 - val_metric_F1score: 0.9957 Epoch 00007: val_acc did not improve from 0.99699 Epoch 8/50 94/94 [==============================] - 0s 4ms/step - loss: 0.7733 - acc: 0.9654 - metric_F1score: 0.9600 - val_loss: 0.6279 - val_acc: 0.9950 - val_metric_F1score: 0.9960 Epoch 00008: val_acc did not improve from 0.99699 Epoch 9/50 94/94 [==============================] - 0s 4ms/step - loss: 0.6846 - acc: 0.9871 - metric_F1score: 0.9746 - val_loss: 0.5987 - val_acc: 0.9950 - val_metric_F1score: 0.9923 Epoch 00009: val_acc did not improve from 0.99699 Epoch 10/50 94/94 [==============================] - 0s 4ms/step - loss: 0.6290 - acc: 0.9858 - metric_F1score: 0.9771 - val_loss: 0.5495 - val_acc: 0.9990 - val_metric_F1score: 0.9985 Epoch 00010: val_acc improved from 0.99699 to 0.99900, saving model to models\multi_hand_gesture_classifier.h5 Epoch 11/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5981 - acc: 0.9891 - metric_F1score: 0.9784 - val_loss: 0.5148 - val_acc: 1.0000 - val_metric_F1score: 0.9995 Epoch 00011: val_acc improved from 0.99900 to 1.00000, saving model to models\multi_hand_gesture_classifier.h5 Epoch 12/50 94/94 [==============================] - 1s 5ms/step - loss: 0.5476 - acc: 0.9898 - metric_F1score: 0.9832 - val_loss: 0.4868 - val_acc: 1.0000 - val_metric_F1score: 0.9995 Epoch 00012: val_acc did not improve from 1.00000 Epoch 13/50 94/94 [==============================] - 0s 4ms/step - loss: 0.5099 - acc: 0.9971 - metric_F1score: 0.9873 - val_loss: 0.4592 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00013: val_acc did not improve from 1.00000 Epoch 14/50 94/94 [==============================] - 0s 4ms/step - loss: 0.4760 - acc: 0.9956 - metric_F1score: 0.9920 - val_loss: 0.4355 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00014: val_acc did not improve from 1.00000 Epoch 15/50 94/94 [==============================] - 0s 5ms/step - loss: 0.4792 - acc: 0.9900 - metric_F1score: 0.9848 - val_loss: 0.4193 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00015: val_acc did not improve from 1.00000 Epoch 16/50 94/94 [==============================] - 0s 5ms/step - loss: 0.4389 - acc: 0.9938 - metric_F1score: 0.9932 - val_loss: 0.4037 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00016: val_acc did not improve from 1.00000 Epoch 17/50 94/94 [==============================] - 0s 5ms/step - loss: 0.4862 - acc: 0.9897 - metric_F1score: 0.9885 - val_loss: 0.3929 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00017: val_acc did not improve from 1.00000 Epoch 18/50 94/94 [==============================] - 0s 5ms/step - loss: 0.4640 - acc: 0.9857 - metric_F1score: 0.9862 - val_loss: 0.4327 - val_acc: 0.9940 - val_metric_F1score: 0.9941 Epoch 00018: val_acc did not improve from 1.00000 Epoch 19/50 94/94 [==============================] - 0s 5ms/step - loss: 0.4202 - acc: 0.9908 - metric_F1score: 0.9906 - val_loss: 0.3634 - val_acc: 1.0000 - val_metric_F1score: 1.0000 Epoch 00019: val_acc did not improve from 1.00000 Epoch 20/50 94/94 [==============================] - 0s 5ms/step - loss: 0.3651 - acc: 0.9986 - metric_F1score: 0.9983 - val_loss: 0.3536 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00020: val_acc did not improve from 1.00000 Epoch 21/50 94/94 [==============================] - 0s 5ms/step - loss: 0.3492 - acc: 0.9982 - metric_F1score: 0.9981 - val_loss: 0.3418 - val_acc: 0.9980 - val_metric_F1score: 0.9980 Epoch 00021: val_acc did not improve from 1.00000
In [ ]:
import matplotlib.pyplot as plt
fig, loss_ax = plt.subplots(figsize=(16, 10))
acc_ax = loss_ax.twinx()
loss_ax.set_xlabel('epoch')
loss_ax.set_ylabel('loss')
acc_ax.plot(history7.history['metric_F1score'], 'b', label='train f1')
acc_ax.plot(history7.history['val_metric_F1score'], 'g', label='val f1')
acc_ax.set_ylabel('f1-score')
acc_ax.legend(loc='upper left')
plt.show()
7. 결론¶
최종 모형으로 선택한 모형은 3가지의 규제 방법을 쓴 모형입니다. 하지만, 위 모형으로 하여 test를 진행하였을 때, 중심에서는 인식을 잘 하지만 영상의 외곽으로 손동작이 갈수록 인식을 잘 못하는 경향이 나타났습니다. 이에, 2편에서는 영상 외곽에서도 손동작을 잘 인식하려면 어떻게 해야하는지에 관한 게시글을 작성하려고 합니다.
감사합니다 :)
'대외활동 > DACrew 2기' 카테고리의 다른 글
| [🔥팀 포스🔥] 첫번째 프로젝트, Multi-Hand Gesture Recognition-2 (0) | 2022.05.27 |
|---|---|
| [ 파이썬으로 만드는 OpenCV 프로젝트🔥] 6장. 영상 필터 (1) | 2022.05.13 |
| [ 파이썬으로 만드는 OpenCV 프로젝트🔥] 5장. 기하학적 변환 (0) | 2022.05.03 |
댓글