01神经网络和深度学习-Planar data classification with one hidden layer v3-第三周编程作业

灰太狼 2022-05-12 09:10 118阅读 0赞

分别使用逻辑回归和加了隐藏层的神经网络对花的颜色进行预测。

## 一、逻辑回归 ##

【注意】这里因为我没有使用IPython，所以显示图像要删掉头部的“**%matplotlib inline**”，加一句“**plt.show()**”，下面同理。

运行结果：

![70][]

![70 1][]

![70 2][]

可以看到，因为图像并不适合拟合线性回归，所以准确率仅为47%。

## 二、神经网络 ##

这里使用了含有隐藏层的神经网络来做预测：

#coding=utf-8
    # Package imports
    import numpy as np
    import matplotlib
    import matplotlib.pyplot as plt
    from testCases import *
    import sklearn
    import sklearn.datasets
    import sklearn.linear_model
    from planar_utils import plot_decision_boundary, sigmoid, load_planar_dataset, load_extra_datasets
    
    np.random.seed(1) # set a seed so that the results are consistent
    
    def layer_sizes(X, Y):
        """
        Arguments:
        X -- input dataset of shape (input size, number of examples)
        Y -- labels of shape (output size, number of examples)
        
        Returns:
        n_x -- the size of the input layer
        n_h -- the size of the hidden layer
        n_y -- the size of the output layer
        """
        ### START CODE HERE ### (≈ 3 lines of code)
        n_x = X.shape[0] # size of input layer
        n_h = 4
        n_y = Y.shape[0] # size of output layer
    
        ### END CODE HERE ###
        return (n_x, n_h, n_y)
    
    # GRADED FUNCTION: initialize_parameters
    def initialize_parameters(n_x, n_h, n_y):
        """
        Argument:
        n_x -- size of the input layer
        n_h -- size of the hidden layer
        n_y -- size of the output layer
        
        Returns:
        params -- python dictionary containing your parameters:
                        W1 -- weight matrix of shape (n_h, n_x)
                        b1 -- bias vector of shape (n_h, 1)
                        W2 -- weight matrix of shape (n_y, n_h)
                        b2 -- bias vector of shape (n_y, 1)
                        """
        
        np.random.seed(2) # we set up a seed so that your output matches ours although the initialization is random.
        
        ### START CODE HERE ### (≈ 4 lines of code)
        W1 = np.random.randn(n_h, n_x) * 0.01 
        b1 = np.zeros((n_h, 1))
        W2 = np.random.randn(n_y, n_h) * 0.01 
        b2 = np.zeros((n_y, 1))
        ### END CODE HERE ###
        
        assert (W1.shape == (n_h, n_x))
        assert (b1.shape == (n_h, 1))
        assert (W2.shape == (n_y, n_h))
        assert (b2.shape == (n_y, 1))
        
        parameters = {"W1": W1,
                      "b1": b1,
                      "W2": W2,
                      "b2": b2}
        
        return parameters
    
        # GRADED FUNCTION: forward_propagation
    def forward_propagation(X, parameters):
        """
        Argument:
        X -- input data of size (n_x, m)
        parameters -- python dictionary containing your parameters (output of initialization function)
        
        Returns:
        A2 -- The sigmoid output of the second activation
        cache -- a dictionary containing "Z1", "A1", "Z2" and "A2"
        """
        # Retrieve each parameter from the dictionary "parameters"
        ### START CODE HERE ### (≈ 4 lines of code)
        W1 = parameters['W1']
        b1 = parameters['b1']
        W2 = parameters['W2']
        b2 = parameters['b2']
        ### END CODE HERE ###
        
        # Implement Forward Propagation to calculate A2 (probabilities)
        ### START CODE HERE ### (≈ 4 lines of code)
        Z1 = W1.dot(X) + b1
        A1 = np.tanh(Z1)
        Z2 = W2.dot(A1) + b2
        A2 = sigmoid(Z2)
        ### END CODE HERE ###
        '''print("Z1 = " + str(Z1))
        print("A1 = " + str(A1))
        print("Z2 = " + str(Z2))
        print("A2 = " + str(A2))'''
        assert(A2.shape == (1, X.shape[1]))
        
        cache = {"Z1": Z1,
                 "A1": A1,
                 "Z2": Z2,
                 "A2": A2}
        
        return A2, cache
    
    # GRADED FUNCTION: compute_cost
    
    def compute_cost(A2, Y, parameters):
        """
        Computes the cross-entropy cost given in equation (13)
        
        Arguments:
        A2 -- The sigmoid output of the second activation, of shape (1, number of examples)
        Y -- "true" labels vector of shape (1, number of examples)
        parameters -- python dictionary containing your parameters W1, b1, W2 and b2
        
        Returns:
        cost -- cross-entropy cost given equation (13)
        """
        
        m = Y.shape[1] # number of example
    
        # Compute the cross-entropy cost
        ### START CODE HERE ### (≈ 2 lines of code)
        logprobs = np.multiply(np.log(A2),Y) + np.multiply(np.log(1 - A2),1 - Y)
        cost = - np.sum(logprobs) / m
        ### END CODE HERE ###
        
        cost = np.squeeze(cost)     # makes sure cost is the dimension we expect. 
                                    # E.g., turns [[17]] into 17 
        assert(isinstance(cost, float))
        
        return cost
    
    # GRADED FUNCTION: backward_propagation
    
    def backward_propagation(parameters, cache, X, Y):
        """
        Implement the backward propagation using the instructions above.
        
        Arguments:
        parameters -- python dictionary containing our parameters 
        cache -- a dictionary containing "Z1", "A1", "Z2" and "A2".
        X -- input data of shape (2, number of examples)
        Y -- "true" labels vector of shape (1, number of examples)
        
        Returns:
        grads -- python dictionary containing your gradients with respect to different parameters
        """
        m = X.shape[1]
        
        # First, retrieve W1 and W2 from the dictionary "parameters".
        ### START CODE HERE ### (≈ 2 lines of code)
        W1 = parameters['W1']
        W2 = parameters['W2']
        ### END CODE HERE ###
            
        # Retrieve also A1 and A2 from dictionary "cache".
        ### START CODE HERE ### (≈ 2 lines of code)
        A1 = cache['A1']
        A2 = cache['A2']
        ### END CODE HERE ###
        
        # Backward propagation: calculate dW1, db1, dW2, db2. 
        ### START CODE HERE ### (≈ 6 lines of code, corresponding to 6 equations on slide above)
        dZ2= A2 - Y
        dW2 = 1.0 / m * dZ2.dot(A1.T)
        db2 = 1.0 / m * np.sum(dZ2, axis = 1, keepdims = True)
        dZ1 = (W2.T).dot(dZ2) * (1 - np.power(A1, 2))
        dW1 = 1.0 / m * dZ1.dot(X.T)
        db1 = 1.0 / m * np.sum(dZ1, axis = 1, keepdims = True)
        ### END CODE HERE ###
        
        grads = {"dW1": dW1,
                 "db1": db1,
                 "dW2": dW2,
                 "db2": db2}
        
        return grads
    
    # GRADED FUNCTION: update_parameters
    
    def update_parameters(parameters, grads, learning_rate = 1.2):
        """
        Updates parameters using the gradient descent update rule given above
        
        Arguments:
        parameters -- python dictionary containing your parameters 
        grads -- python dictionary containing your gradients 
        
        Returns:
        parameters -- python dictionary containing your updated parameters 
        """
        # Retrieve each parameter from the dictionary "parameters"
        ### START CODE HERE ### (≈ 4 lines of code)
        W1 = parameters['W1']
        b1 = parameters['b1']
        W2 = parameters['W2']
        b2 = parameters['b2']
        ### END CODE HERE ###
        
        # Retrieve each gradient from the dictionary "grads"
        ### START CODE HERE ### (≈ 4 lines of code)
        dW1 = grads['dW1']
        db1 = grads['db1']
        dW2 = grads['dW2']
        db2 = grads['db2']
        ## END CODE HERE ###
        
        # Update rule for each parameter
        ### START CODE HERE ### (≈ 4 lines of code)
        W1 = W1-learning_rate * dW1
        b1 = b1-learning_rate * db1
        W2 = W2-learning_rate * dW2
        b2 = b2-learning_rate * db2
        
        ### END CODE HERE ###
        
        parameters = {"W1": W1,
                      "b1": b1,
                      "W2": W2,
                      "b2": b2}
        
        return parameters
     
    # GRADED FUNCTION: nn_model
    
    def nn_model(X, Y, n_h, num_iterations = 10000, print_cost=False):
        """
        Arguments:
        X -- dataset of shape (2, number of examples)
        Y -- labels of shape (1, number of examples)
        n_h -- size of the hidden layer
        num_iterations -- Number of iterations in gradient descent loop
        print_cost -- if True, print the cost every 1000 iterations
        
        Returns:
        parameters -- parameters learnt by the model. They can then be used to predict.
        """
        
        np.random.seed(3)
        n_x = layer_sizes(X, Y)[0]
        n_y = layer_sizes(X, Y)[2]
        
        # Initialize parameters, then retrieve W1, b1, W2, b2. Inputs: "n_x, n_h, n_y". Outputs = "W1, b1, W2, b2, parameters".
        ### START CODE HERE ### (≈ 5 lines of code)
        parameters = initialize_parameters(n_x, n_h, n_y)
        W1 = parameters["W1"]
        b1 = parameters["b1"]
        W2 = parameters["W2"]
        b2 = parameters["b2"]
        ### END CODE HERE ###
        
        # Loop (gradient descent)
    
        for i in range(0, num_iterations):
             
            ### START CODE HERE ### (≈ 4 lines of code)
            # Forward propagation. Inputs: "X, parameters". Outputs: "A2, cache".
            A2, cache = forward_propagation(X, parameters)
            
            # Cost function. Inputs: "A2, Y, parameters". Outputs: "cost".
            cost = compute_cost(A2, Y, parameters)
     
            # Backpropagation. Inputs: "parameters, cache, X, Y". Outputs: "grads".
            grads = backward_propagation(parameters, cache, X, Y)
     
            # Gradient descent parameter update. Inputs: "parameters, grads". Outputs: "parameters".
            parameters = update_parameters(parameters, grads)
            
            ### END CODE HERE ###
            
            # Print the cost every 1000 iterations
            if print_cost and i % 1000 == 0:
                print ("Cost after iteration %i: %f" %(i, cost))
    
        return parameters
     
    # GRADED FUNCTION: predict
    
    def predict(parameters, X):
        """
        Using the learned parameters, predicts a class for each example in X
        
        Arguments:
        parameters -- python dictionary containing your parameters 
        X -- input data of size (n_x, m)
        
        Returns
        predictions -- vector of predictions of our model (red: 0 / blue: 1)
        """
        
        # Computes probabilities using forward propagation, and classifies to 0/1 using 0.5 as the threshold.
        ### START CODE HERE ### (≈ 2 lines of code)
        A2, cache = forward_propagation(X, parameters)
        predictions = np.rint(A2)
        ### END CODE HERE ###
        
        return predictions
    
    if __name__=='__main__':
        X, Y = load_planar_dataset()
        
        X_assess, Y_assess = layer_sizes_test_case()
        (n_x, n_h, n_y) = layer_sizes(X_assess, Y_assess)
        print("The size of the input layer is: n_x = " + str(n_x))
        print("The size of the hidden layer is: n_h = " + str(n_h))
        print("The size of the output layer is: n_y = " + str(n_y))
    
        n_x, n_h, n_y = initialize_parameters_test_case()
    
        parameters = initialize_parameters(n_x, n_h, n_y)
        print("W1 = " + str(parameters["W1"]))
        print("b1 = " + str(parameters["b1"]))
        print("W2 = " + str(parameters["W2"]))
        print("b2 = " + str(parameters["b2"]))
        
        X_assess, parameters = forward_propagation_test_case()
        A2, cache = forward_propagation(X_assess, parameters)
        # Note: we use the mean here just to make sure that your output matches ours. 
        print(np.mean(cache['Z1']) ,np.mean(cache['A1']),np.mean(cache['Z2']),np.mean(cache['A2']))
     
        A2, Y_assess, parameters = compute_cost_test_case()
        print("cost = " + str(compute_cost(A2, Y_assess, parameters)))
    
        parameters, cache, X_assess, Y_assess = backward_propagation_test_case()
        grads = backward_propagation(parameters, cache, X_assess, Y_assess)
        print ("dW1 = "+ str(grads["dW1"]))
        print ("db1 = "+ str(grads["db1"]))
        print ("dW2 = "+ str(grads["dW2"]))
        print ("db2 = "+ str(grads["db2"]))
    
        parameters, grads = update_parameters_test_case()
        parameters = update_parameters(parameters, grads)
        print("W1 = " + str(parameters["W1"]))
        print("b1 = " + str(parameters["b1"]))
        print("W2 = " + str(parameters["W2"]))
        print("b2 = " + str(parameters["b2"]))
    
        X_assess, Y_assess = nn_model_test_case()
        parameters = nn_model(X_assess, Y_assess, 4, num_iterations=10000, print_cost=False)
        print("W1 = " + str(parameters["W1"]))
        print("b1 = " + str(parameters["b1"]))
        print("W2 = " + str(parameters["W2"]))
        print("b2 = " + str(parameters["b2"]))
    
        parameters, X_assess = predict_test_case()
        predictions = predict(parameters, X_assess)
        print("predictions mean = " + str(np.mean(predictions)))
    
        # Build a model with a n_h-dimensional hidden layer
        parameters = nn_model(X, Y, n_h = 4, num_iterations = 10000, print_cost=True)
        # Plot the decision boundary
        plot_decision_boundary(lambda x: predict(parameters, x.T), X, Y)
        plt.title("Decision Boundary for hidden layer size " + str(4))
    
        # Print accuracy
        predictions = predict(parameters, X)
        print ('Accuracy: %d' % float((np.dot(Y,predictions.T) + np.dot(1-Y,1-predictions.T))/float(Y.size)*100) + '%')
    
        # This may take about 2 minutes to run
        plt.figure(figsize=(16, 32))
        hidden_layer_sizes = [1, 2, 3, 4, 5, 20, 40]
        for i, n_h in enumerate(hidden_layer_sizes):
            plt.subplot(5, 2, i+1)
            plt.title('Hidden Layer of size %d' % n_h)
            parameters = nn_model(X, Y, n_h, num_iterations = 5000)
            plot_decision_boundary(lambda x: predict(parameters, x.T), X, Y)
            predictions = predict(parameters, X)
            accuracy = float((np.dot(Y,predictions.T) + np.dot(1-Y,1-predictions.T))/float(Y.size)*100)
            print ("Accuracy for {} hidden units: {} %".format(n_h, accuracy))
    
        # Datasets
        noisy_circles, noisy_moons, blobs, gaussian_quantiles, no_structure = load_extra_datasets()
        datasets = {"noisy_circles": noisy_circles,
                "noisy_moons": noisy_moons,
                "blobs": blobs,
                "gaussian_quantiles": gaussian_quantiles}
        ### START CODE HERE ### (choose your dataset)
        dataset = "noisy_moons"
        ### END CODE HERE ###
        
        X, Y = datasets[dataset]
        X, Y = X.T, Y.reshape(1, Y.shape[0])
        # make blobs binary
        if dataset == "blobs":
            Y = Y%2
    
        # Visualize the data
        plt.scatter(X[0, :], X[1, :], c=Y, s=40, cmap=plt.cm.Spectral);
        plt.show()

【注意1】在步骤进行到对于**4.3 - The Loop** 部分的 “**def forward\_propagation(X, parameters)**”函数时，因为我的代码跑出来的结果和expected不同，找bug找了很久，最后实在找不出来就硬着头皮接着敲下面的部分，居然发现后面的结果都和expected相同……(｡•́︿•̀｡)大胆猜测一下，是不是原expected出错了呢……？

（1）原expected：

![70 3][]

（2）我的output：

![70 4][]

【注意2】预测值四舍五入的时候，最好不要使用“**np.round()**”，而是使用“**np.rint()**”，[原因][Link 1]。

【注意3】后面我是测试了“**hidden\_layer\_sizes = \[1, 2, 3, 4, 5, 20, 40\]**”这几种不同隐藏层层数的神经网络，，预测准确率提高到了90%以上。当层数增大的时候，抛出 Memory Error 超出内存。

运行结果：

![70 5][]

![70 6][]

![70 7][]

[70]: /images/20220512/821aa0f1bdfa403daf2af1d4b241bf22.png
[70 1]: /images/20220512/213c31bab2b5455190992a71de16e97f.png
[70 2]: /images/20220512/c1ff1cc9c4ac41d98701c59c284aa570.png
[70 3]: /images/20220512/604cfc11bfbd4a6c9bf92e4b0f3ba794.png
[70 4]: /images/20220512/cfaa6d1f1bbf4c1e81e6789f3c4a993c.png
[Link 1]: https://www.cnblogs.com/xiangnan/p/3430579.html
[70 5]: /images/20220512/755ef13632dd4b47882b9e9e38d1fd0d.png
[70 6]: /images/20220512/7ab37edb77354148801f8910c17ceb8b.png
[70 7]: /images/20220512/3b2ba3227f324df29c21173e83a06564.png