compute_delta_L_per_data_point()

home > kero > Documentation

Compute the value of \delta^L shown in Neural Network and Back Propagation.

kero.multib.NeuralNetwork.py

class NetworkUpdater:
  def compute_delta_L_per_data_point(self, z_L, Y0, Y_set, Y0_set, AF,
      verbose=False,
      print_format="%6.8f"):
    return delta_L

Arguments/Return

z_L numpy matrix. Vector of size m x 1, where m is the number of neurons in the output layer. In the neural network this is the values at the output layer before activation function.
Y0 numpy matrix. Observed/true output data. Vector of size m x 1, where m is the number of neurons in the output layer.
Y_set List of numpy matrix, [Y]. Each numpy matrix Y is a column vector. In a neural network, this is the values at the output layer predicted by the neural network.
Y0_set List of numpy matrix, [Y0]. Each numpy matrix Y is a column vector. There should be equal number of Y in Y_set as Y0 in Y0_set.
AF AF (activationFunction). Assume it is initiated.
verbose Bool False or integer

The larger the integer, the more information is printed. Set them to suitable integers for debugging.

Default=False

print_format String. Format for printing numpy matrices when verbose is beyond some value.

Default=”%6.8f”

return delta_L numpy matrix. Vector of size m x 1 where m is the number of neurons in layer L or output layer.

Example Usage 1

Note that in order to use the following script, we need prep() function which is available in Deep Learning and Neural Network with kero PART 1.

testNNupdater2A.py

import testNNupdater2aux as taux
import kero.multib.NeuralNetwork as nn
import kero.utils.utils as ut
import numpy as np

print("--- test2A ---")

# input_set : list of numpy matrix. 
# Y_set : list of numpy matrix. Output computed by NN
# Y0_set : list of numpy matrix. True/observed output
#  the grand objective is to train NN so that Y_set is equal to Y0_set 
# -------------------------------------------
# this is a collection of a_l_set and z_l_sedbt over all data points
#   z_l_set is the collection of z values over all layers, l=2,3,...L
#   and a_l_set is the corresponding activated values
#   Recall: a_l_set and z_l_set each is a list of numpy matrices 

out = taux.prep(print_data=False)
input_set=out["input_set"]
Y_set=out["Y_set"]
Y0_set=out["Y0_set"]
collection_of_fed_forward_a_l=out["collection_of_fed_forward_a_l"]
collection_of_fed_forward_z_l=out["collection_of_fed_forward_z_l"]
weights=out["weights"]
biases=out["biases"]
NeuralNetwork=out["NeuralNetwork"]

a_L_set = Y_set
# weights = [w_2,w_3,w_4]
# biases = [b_2,b_3,b_4]


nu = nn.NetworkUpdater()
AF = nn.activationFunction(func = "Sigmoid")
for i in range(len(input_set)): # for each data point
	z_l_set = collection_of_fed_forward_z_l[i]

	print("data point ", i," : ")
	# print("  z_L:")
	# ut.print_numpy_matrix(z_L,formatting="%6.2f",no_of_space=10)
	# print("  Y0:")
	# ut.print_numpy_matrix(Y0,formatting="%6.2f",no_of_space=10)
	
	# l = L = 4
	z_L = z_l_set[-1 - 0]
	Y0 = Y0_set[i]
	delta_L = nu.compute_delta_L_per_data_point(z_L, Y0, a_L_set, Y0_set, AF,
			verbose=11,
			print_format="%6.8f")

	# l = 3
	delta_l_plus_1 = delta_L
	w_l_plus_1 = weights[- 1 -0] # w_4
	z_l = z_l_set[-1 -1]
	delta_l = nu.compute_delta_l_per_data_point(w_l_plus_1, delta_l_plus_1, z_l, AF,
		verbose=11,
		print_format="%6.8f")

	# l = 2
	delta_l_plus_1 = delta_l
	w_l_plus_1 = weights[- 1 -1] # w_4
	z_l = z_l_set[-1 -2]
	delta_l = nu.compute_delta_l_per_data_point(w_l_plus_1, delta_l_plus_1, z_l, AF,
		verbose=11,
		print_format="%6.8f")
	# print("------------------------------------------")

We have prepared 24 data points using prep(), and the following shows the output for the first 2 data points.

---aux---
Initializing a Neural Network object.
--- test2A ---
Initializing a Neural Network object.
data point  0  :
 -+ compute_delta_L_per_data_point()
    delta_L =
          0.11825302
          0.12284915
 -+ compute_differential_term_at_l().
    np.array(w_l_plus_1).shape =  (2, 2)
    delta_l =
          0.00598236
          0.00598236
 -+ compute_differential_term_at_l().
    np.array(w_l_plus_1).shape =  (2, 3)
    delta_l =
          0.00029088
          0.00025446
          0.00029721
data point  1  :
 -+ compute_delta_L_per_data_point()
    delta_L =
          0.11825325
          0.12284961
 -+ compute_differential_term_at_l().
    np.array(w_l_plus_1).shape =  (2, 2)
    delta_l =
          0.00598275
          0.00598275
 -+ compute_differential_term_at_l().
    np.array(w_l_plus_1).shape =  (2, 3)
    delta_l =
          0.00029146
          0.00025704
          0.00029734
...

kero version: 0.6.3