Sunday, March 29, 2020

[Deep Learning by Keras] 01. How to make Linear regression for 5 minutes


Hello. I'm DVM who is talk about deep learning for 5minutes using keras
Today I'm going to talk about how to make linear regression


wiki pedia saids linear regression is 
the relationships are modeled using linear predictor functions 
whose unknown model parameters are estimated from the data

Let's make model
Code is very simple.
define the sequential model and add the layer.
you could adjust the number of the learning parameters.


Linear regression is learning the weight and bias like equation untill cost function is minimized.
we need to adjust learning rate. 

If learning rate is big, cost funtion become divergence
if learning rate is too small, cost function fall into local minimum which is not optimization.

we adjust learning rate untill cost values is small.
It is very easy.

Let`s practice coding using keras

01__5minute_keras_linear_regression

Let's define library

  • Tensorflow include keras
In [0]:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras import optimizers
Let's define train datas and test datas
  • we have to train using train datas
  • we evaluate using test datas
In [0]:
x_train = [1,2,3,4]
y_train = [2,3,4,5]
x_test =  [5,6,7]
y_test =  [6,7,8]

Let's make a single model

In [0]:
model = Sequential()
model.add(Dense(1,input_dim=1))

Let's learn using stochastic gradient decent

  • epochs means how many times train the datas
In [0]:
sgd = optimizers.SGD(lr=0.1)
model.compile(loss='mse',optimizer=sgd)
model.summary()
model.fit(x_train,y_train,epochs=30)

Let's check the predicted datas

In [0]:
y_predict = model.predict(x_test)
print('prediction:\n',y_predict)

Let's evaluate using test datas

In [0]:
score = model.evaluate(x_test,y_test)
print('cost',score)

Total codes

In [23]:
from tensorflow.keras.models import  Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras import optimizers
import numpy as np
x_train = [1,2,3,4]
y_train = [2,3,4,5]
x_test = [5,6,7]
y_test = [6,7,8]
model = Sequential()
model.add(Dense(1,input_dim=1)) 
sgd = optimizers.SGD(lr = 0.1)
model.compile(loss='mse',optimizer =sgd)
model.summary()
model.fit(x_train,y_train,epochs=30)
y_predict = model.predict(x_test)
print('Prediction : \n',y_predict)
score = model.evaluate(x_test,y_test)
print('cost',score)
Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_3 (Dense)              (None, 1)                 2         
=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________
Epoch 1/30
1/1 [==============================] - 0s 2ms/step - loss: 20.9350
Epoch 2/30
1/1 [==============================] - 0s 1ms/step - loss: 9.4192
Epoch 3/30
1/1 [==============================] - 0s 1ms/step - loss: 4.2477
Epoch 4/30
1/1 [==============================] - 0s 1ms/step - loss: 1.9246
Epoch 5/30
1/1 [==============================] - 0s 1ms/step - loss: 0.8806
Epoch 6/30
1/1 [==============================] - 0s 1ms/step - loss: 0.4109
Epoch 7/30
1/1 [==============================] - 0s 4ms/step - loss: 0.1991
Epoch 8/30
1/1 [==============================] - 0s 1ms/step - loss: 0.1031
Epoch 9/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0592
Epoch 10/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0388
Epoch 11/30
1/1 [==============================] - 0s 2ms/step - loss: 0.0288
Epoch 12/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0237
Epoch 13/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0208
Epoch 14/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0189
Epoch 15/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0174
Epoch 16/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0163
Epoch 17/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0153
Epoch 18/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0143
Epoch 19/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0135
Epoch 20/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0127
Epoch 21/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0119
Epoch 22/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0112
Epoch 23/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0106
Epoch 24/30
1/1 [==============================] - 0s 3ms/step - loss: 0.0099
Epoch 25/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0093
Epoch 26/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0088
Epoch 27/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0083
Epoch 28/30
1/1 [==============================] - 0s 2ms/step - loss: 0.0078
Epoch 29/30
1/1 [==============================] - 0s 2ms/step - loss: 0.0073
Epoch 30/30
1/1 [==============================] - 0s 1ms/step - loss: 0.0069
Prediction : 
 [[6.1380534]
 [7.2050886]
 [8.272123 ]]
1/1 [==============================] - 0s 1ms/step - loss: 0.0451
cost 0.04505706951022148