Lab 16: Classification III: Neural Networks and Deep Learning

Lab 16: Classification III: Neural Networks and Deep Learning#

For the class on Wednesday, April 2nd

No code needs to be run in this notebook.

You can still download this notebook and write your answers in this notebook, and submit it as usual. Alternatively, you can directly submit your answers on Canvas, either as plain text or a PDF file.

A. Neural Network Playground (in-class activity)#

🚀 Tasks and Questions:

Go to this link. Try the following changes and document what you observe. For each case, briefly (2-3 sentences) discuss your observations (e.g., Do they make sense? Are they surprising in any way? What do you learn from them?)

  1. Remove one neuron.

  2. Try different learning rates (add back the neuron you removed in Step 1 first).

    • Change learning rate to 0.003

    • Change learning rate to 0.3

    • Change learning rate to 3

  3. Try the sigmoid activation function (set learning rate back to 0.03).

  4. Try L2 regularization with regularization rate of 0.1 (set activation back to ReLU).

Add your answers to Part A here:

B. Text Classification with TensorFlow#

🚀 Tasks:

We will follow the example in this notebook to do a text classification example using TensorFlow.

You are encouraged to run the tutorial notebook on Google Colab. However, running the notebook is not required to answer the following questions since the link above provides a notebook that has been ran already.

If you are running the tutorial notebook on Google Colab, you will need to change the runtime type to “T4 GPU”. In addition, you will need to add a line at the beginning of the notebook before you start running.

%env TF_USE_LEGACY_KERAS=1

The example uses three layers in total: two hidden layers and one output layer. The first hidden layer is a pre-trained text embedding layer that projects input text into a 50-dimensional space.

đź“ť Questions:

  1. How many instances of data in total were used in this notebook? How many were used in the test set? How many were used in the validation set?

  2. How many neurons are in the second hidden layer? How many neurons are in the third layer (the output layer)?

  3. There are 816 parameters in the second layer, and 17 in the third layer. Explain where the numbers 816 and 17 come from.

  4. The model is trained with 40 epochs, and there are 30 steps in each epoch. Explain where the number 30 comes from.

  5. From the two plots near the end, we observe that the “early stopping” method should help prevent overfitting. What would be a good epoch to stop?

  6. The two plots show that, for the validation set, the loss keeps increasing as the epoch increases, but the accuracy roughly stays constant. What do their different behaviors indicate?

Add your answers to Part B here:

If you write your answers in this notebook, please submit it following the usual instructions. You can also directly submit your answers on Canvas, either as plain text or a PDF file.

Tip

Submit your notebook

Follow these steps when you complete this lab and are ready to submit your work to Canvas:

  1. Check that all your text answers, plots, and code are all properly displayed in this notebook.

  2. Run the cell below.

  3. Download the resulting HTML file 16.html and then upload it to the corresponding assignment on Canvas.

!jupyter nbconvert --to html --embed-images 16.ipynb