[Tensorflow] MNIST DNN으로 구현해보기

In [1]:

import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten, Dense, Dropout
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split

In [2]:

mnist = pd.read_csv('/content/drive/MyDrive/MultiCampus/data/mnist/train.csv')
mnist.head()

Out[2]:

	label	pixel0	pixel1	pixel2	pixel3	pixel4	pixel5	pixel6	pixel7	pixel8	...	pixel774	pixel775	pixel776	pixel777	pixel778	pixel779	pixel780	pixel781	pixel782	pixel783
0	1	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
1	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
2	1	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
3	4	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
4	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0

5 rows × 785 columns

In [3]:

img_data = mnist.drop('label', axis=1, inplace=False).values

fig = plt.figure()
for i in range(25):
    ax = fig.add_subplot(5, 5, i+1)
    ax.imshow(img_data[i].reshape(28,28), cmap='Greys', interpolation='nearest')

plt.tight_layout()
plt.show()

Test Data Set, Validation Data Set 분할¶

In [4]:

# 데이터 분할
train_x, test_x, train_y, test_y = \
train_test_split(mnist.drop('label', axis=1, inplace=False), 
                 mnist['label'],
                 test_size=0.2,
                 random_state=1,
                 stratify=mnist['label'])

현재 x데이터의 값이 0~255의 분포를 가진다.
0~1의 분포를 가지도록 스케일링 해준다.
이 데이터는 결측치나 이상치가 없기때문에
MinMaxScaler를 이용하지않고 /255를 해주는 경우도 있다.(ex. Tensorflow Tutorial)

In [5]:

# 정규화
scaler = MinMaxScaler()
scaler.fit(train_x)
norm_train_x = scaler.transform(train_x)
norm_test_x = scaler.transform(test_x)

sparse catergorical cross entropy를 이용하면¶

one_hot encoding은 하지 않아도 된다.¶

In [7]:

model = Sequential()

model.add(Flatten(input_shape=(784,)))
model.add(Dense(units=256, activation='relu')) # sigmoid는 gradient vanishing의 위험이 있음
model.add(Dropout(0.1))
model.add(Dense(units=128, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(units=10, activation='softmax'))

model.compile(optimizer=SGD(learning_rate=1e-3),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

################### EarlyStopping #########################
es = EarlyStopping(monitor='val_loss', # monitoring  값
                   min_delta=0.001, # threshold
                   patience=5, # 참는 횟수
                   mode='auto',
                   restore_best_weights=True)

################## Model Save ############################
ckpt = ModelCheckpoint(filepath='/content/drive/MyDrive/MultiCampus/data/mnist/mnist.ckpt',
                       save_weights_only = True)

model.summary()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten (Flatten)           (None, 784)               0         
                                                                 
 dense (Dense)               (None, 256)               200960    
                                                                 
 dropout (Dropout)           (None, 256)               0         
                                                                 
 dense_1 (Dense)             (None, 128)               32896     
                                                                 
 dropout_1 (Dropout)         (None, 128)               0         
                                                                 
 dense_2 (Dense)             (None, 10)                1290      
                                                                 
=================================================================
Total params: 235,146
Trainable params: 235,146
Non-trainable params: 0
_________________________________________________________________

In [8]:

# 학습
history = model.fit(norm_train_x,
                    train_y, 
                    epochs=5000,
                    batch_size=100,
                    validation_split=0.2,
                    verbose=0, 
                    callbacks=[es, ckpt])

Colab TPU 이용 => 9분 소요

Test Data Accuracy¶

In [10]:

result = model.evaluate(norm_test_x, test_y)
print(result)

263/263 [==============================] - 1s 3ms/step - loss: 0.1698 - accuracy: 0.9530
[0.16975967586040497, 0.9529761672019958]

Train Data Accuracy¶

In [11]:

model.evaluate(norm_train_x, train_y)

1050/1050 [==============================] - 5s 4ms/step - loss: 0.1367 - accuracy: 0.9616

Out[11]:

[0.13672791421413422, 0.9615773558616638]

CNN을 이용하거나,
Tensorflow 제공 데이터셋(추가 데이터셋)을 이용해서 추가학습을 진행하면
정확도를 높일 수 있다.