Ctrl+K

Lab 16: Classification III: Neural Networks and Deep Learning

Lab 16: Classification III: Neural Networks and Deep Learning#

For the class on Monday, April 1st

No code needs to be run in this notebook. You can still download the notebook and write your answers in this notebook, and submit it as usual. Alternatively, you can directly submit your answers on Canvas, either as plain text or a PDF file.

A. Neural Network Playground (in class)#

Go to this link. Try the following changes and document what you observe.

  1. Remove one neuron

  2. Try different learning rates (add back the neuron you removed in Step 1)

    • Change learning rate to 0.003

    • Change learning rate to 0.3

    • Change learning rate to 3

  3. Try the sigmoid activation function (set learning rate back to 0.03)

  4. Try L2 regularization with regularization rate of 0.1 (set activation back to ReLU)

Add your answer to Part A here:

B. Text Classification with TensorFlow#

We will follow the example in this notebook to do a text classification example using TensorFlow.

You are encouraged to run the notebook on Google Colab; however, running the notebook is not required to answer the following questions.

The example uses three layers in total: two hidden layers and one output layer. The first hidden layer is a pre-trained text embedding layer that projects input text into a 50-dimensional space.

  1. How many instances of data in total were used in this notebook? How many were used in the test set? How many were used in the validation set?

  2. How many neurons are in the second hidden layer? How many neurons are in the third layer (the output layer)?

  3. There are 816 parameters in the second layer, and 17 in the third layer. Explain where the numbers 816 and 17 come from.

  4. The model is trained with 40 epochs, and there are 30 steps in each epoch. Explain where the number 30 comes from.

  5. From the two plots near the end, we observe that the “early stopping” method should help prevent overfitting. What would be a good epoch to stop?

  6. The two plots show that, for the validation set, the loss keeps increasing as the epoch increases, but the accuracy roughly stays constant. What do their different behaviors indicate?

Add your answer to Part B here:

C. Final project progress#

Give a brief (about 2-3 sentences) update on your final project. You can mention what methods you have tried, any issues you have encountered, and/or any preliminary results you have.

Add your answer to Part C here:

You can submit this notebook as usual, or you can directly submit your answers on Canvas, either as plain text or a PDF file.

Tip

How to submit this notebook on Canvas?

  1. Make sure all your answers, code, and desired results are properly displayed in the notebook.

  2. Save the notebook (press Ctrl+s or Cmd+s). The grey dot on the filename tab (indicating “unsaved”) should disappear.

  3. Run the following cell.

  4. Upload the resulting HTML file to Canvas under the corresponding assignment.

! jupyter nbconvert --to html ./16.ipynb
Contents