python - Designing an Efficient neural network for a Regression data -
by following tutorials on tensorflow , reading basic stuff neural networks, have modeled neural network using python , tensorflow libraries.
as of now,my ".csv" file data follows:
at v ap rh pe 14.96 41.76 1024.07 73.17 463.26 25.18 62.96 1020.04 59.08 444.37 5.11 39.4 1012.16 92.14 488.56 20.86 57.32 1010.24 76.64 446.48 10.82 37.5 1009.23 96.62 473.9 26.27 59.44 1012.23 58.77 443.67 15.89 43.96 1014.02 75.24 467.35 9.48 44.71 1019.12 66.43 478.42 14.64 45 1021.78 41.25 475.98 .....................................
as of now,i have designed neural network function multiple inputs , multiple outputs. in above data,i considering first 3 columns inputs , next 2 columns outputs.so,once training data,if pass inputs 14.64,45,1021.78 ,i want neural network predict output values 41.25 , 475.98. here current code:
import tensorflow tf import numpy np import pandas pd #import matplotlib.pyplot plt rng = np.random # parameters learning_rate = 0.01 training_epochs = 5000 display_step = 1000 batch_size = 100 # read data csv df = pd.read_csv("h:\minithessis\sample.csv") # in[173]: # seperating out dependent & independent variable train_x = df[['at','v','ap']] train_y = df[['rh','pe']] trainx = train_x.as_matrix().astype(np.float32) trainy = train_y.as_matrix().astype(np.float32) n_input = 3 n_classes = 2 n_hidden_1 = 20 n_hidden_2 = 20 n_samples = len(trainx) # tf graph input #inserts placeholder tensor fed. x = tf.placeholder(tf.float32, [none, n_input]) y = tf.placeholder(tf.float32, [none, n_classes]) # set model weights w_h1 = tf.variable(tf.random_normal([n_input, n_hidden_1])) w_h2 = tf.variable(tf.random_normal([n_hidden_1, n_hidden_2])) w_out = tf.variable(tf.random_normal([n_hidden_2, n_classes])) b_h1 = tf.variable(tf.zeros([n_hidden_1])) b_h2 = tf.variable(tf.zeros([n_hidden_2])) b_out = tf.variable(tf.zeros([n_classes])) # in[175]: # construct linear model layer_1 = tf.add(tf.matmul(x, w_h1), b_h1) layer_1 = tf.nn.relu(layer_1) layer_2 = tf.add(tf.matmul(layer_1, w_h2), b_h2) layer_2 = tf.nn.relu(layer_2) out_layer = tf.matmul(layer_2, w_out) + b_out # in[176]: # mean squared error #cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=out_layer, labels=y)) cost = tf.reduce_mean(tf.pow(out_layer-y, 2))/(2*n_samples) #cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=out_layer, labels=y)) # gradient descent optimizer = tf.train.adamoptimizer(learning_rate).minimize(cost) # in[177]: # initializing variables init = tf.global_variables_initializer() # in[181]: initialval = 0 finalval = 100 batchcount = int(n_samples/100) remainder = n_samples%100 print(remainder) if remainder!=0: batchcount = batchcount +1 # launch graph tf.session() sess: sess.run(init) # fit training data epoch in range(training_epochs): avg_cost = 0. batchidx in range(batchcount): if remainder != 0 , batchidx==batchcount-1: finalval = finalval-(100-remainder) subtrainx = trainx[initialval:finalval] subtrainy = trainy[initialval:finalval] _, c = sess.run([optimizer, cost], feed_dict={x: subtrainx,y: subtrainy}) initialval = initialval+100 finalval = finalval+100 avg_cost += c/batchcount # display logs per epoch step if epoch % display_step == 0: print("epoch:", '%04d' % (epoch+1), "cost=", \ "{:.9f}".format(avg_cost)) #print("optimization finished!") #training_cost = sess.run(cost, feed_dict={x: trainx,y: trainy}) #print(training_cost) best = sess.run([out_layer], feed_dict={x: np.array([[14.96,41.76,1024.07]])}) print(best)
the architecture of neural network follows: 1) no. of input nodes(n_inputs) , output(n_classes) nodes 3 , 2 respectively 2)as of now,i considering 2 hidden layers ,each having 20 nodes
i need regarding in following points:
1) how select parameters "training_epoch" ; "learning_rate" ; "batch_size" here,so can have better accuracy?
2)i still not sure architecture of neural network. how many hidden layers recommended use? , also, number of nodes in each hidden layer?
3)if suppose,in data,i want use first 2 columns inputs , next 3 columns outputs,then changes,i can make? need change complete architecture?
4)also,i not sure cost function.which 1 better use better accuracy?
5) also,please let me know,if missing important parameter,which worth considering!
thanks in advance!
the questions asking quite general, , if universal answer discovered, data scientists cease exist, since process of machine learning system construction automated.
1,2,4. learning rate , epoch count correspondingly: "the less better" , "the more better". in practice however, need machine finish education before heat death of universe sets in.
usually, @ values of error function network score on dataset train on. @ first network learning data , score gets better each epoch. after while, network learns dataset , score stops improving. after there's no point of continuing. may quit earlier if error small enough uses.
the learning rate can chosen on basis of evolution of error. higher rate is, faster network finish learning, if high, network can overshoot optimal value of parameters worse , worse on time. need find compromise between speed of learning , accuracy. not uncommon reduce learning rate epoch counter increases.
you've used word "accuracy" need first define means in terms of problem. (it tightly related cost function, actually.) problem you've described falls class of linear\nonlinear regression (as opposed logistic regression), these problems mean square error (which used in code) customary. cross-entropy criteria more suited classification problems, usually. if have domain expertise data of course may devise specialized cost functions network.
there no single answer best structure of network. it's complexity depends on function trying approximate. if connection between input , output data linear, don't need more single neuron. generally, task of deciding on structure of network optimization problem itself. so, first set aside test data answers, train network on rest of data, see how performs on test data, has not seen before. test network, modify it, train it, , test again. based on performance of network on both training , test sets decide how alter next.
3. need change number of inputs , outputs. , have train network anew.
also, these , many more basic questions covered in many courses on internet, consider taking those. 1 of such courses can found here.
Comments
Post a Comment