Machine Learning: Linear Regression - Revision history

Aplstudent at 18:09, 12 November 2016

2016-11-12T18:09:49Z

Aplstudent at 08:10, 12 November 2016

2016-11-12T08:10:54Z

Aplstudent at 08:08, 12 November 2016

2016-11-12T08:08:46Z

Aplstudent at 08:06, 12 November 2016

2016-11-12T08:06:19Z

Aplstudent at 08:05, 12 November 2016

2016-11-12T08:05:34Z

Aplstudent at 08:04, 12 November 2016

2016-11-12T08:04:36Z

Aplstudent: Created page with "Machine learning is the subfield of Artificial intelligence interested in making agents that review their past actions and experiences to formulate a new action. The task is t..."

2016-11-12T07:58:48Z

Created page with "Machine learning is the subfield of Artificial intelligence interested in making agents that review their past actions and experiences to formulate a new action. The task is t..."

New page

Machine learning is the subfield of Artificial intelligence interested in making agents that review their past actions and experiences to formulate a new action. The task is to learn the formula for a function. A performance standard is used to adjust the function that is predicted. Linear regression is a common pattern that can be approximated by learning the parameters for slope and intercept.

Here are the packages that are used and three points to approximate a function from.
<code>
import numpy as np
import matplotlib.pyplot as plt
pair=np.array([(1,1),(4,4.2),(2,2)])

</code>

[[File:Scatterlinearpost.png]]

The slope and intercept must be initialized by the user. In addition the learning rate is how quickly to change the parameters by each iteration. the data set is fitted multiple times with updated starting weights, these are epochs.

<code>

nepoch = 4
m=0.0
b=0.0
alpha = 0.2

</code>

each pair is looped through and a y is predicted according to <math>Y_pred=X*m+b</math> The error is then found according to <math>Y_pred-Y</math> The weights are then updated according to the formula <math>m:=m-learning_rate*error*x</math> and <math>b:=b-learning_rate*error</math> The new value of the paramaters are used with the next set of points.

<code>

for i in range(0,nepoch,1):
print("i=" , i)
for i in [0,1,2]:
print(m,b)
x=pair[i][0]
y=pair[i][1]
y_pred=m*x+b
error=y_pred-y
m=m-alpha*error*x
b=b-alpha*error

print(m,b)
</code>

the slope is found to be 1.04 and the y-intercept as -0.08. The exact values were not found because this algorithm approximates the local minimum. It overshoots the exact value and then updates it by overshooting it again resulting in the answer revolving around the exact value.

← Older revision		Revision as of 18:09, 12 November 2016
Line 63:		Line 63:
	This is a graph of the final parameters,an appropriate answer was found:		This is a graph of the final parameters,an appropriate answer was found:
	[[File:Linearregression2tut.png]]		[[File:Linearregression2tut.png]]
		+
		+
		+
		+
		+	sources:
		+	Artificial Intelligence : A modern Aproach by Stuart Russel and Peter Norvig, chapter 19.
		+	http://machinelearningmastery.com/linear-regression-tutorial-using-gradient-descent-for-machine-learning/

@@ Line 22: / Line 22: @@
 [[File:Scatterlinearpost.png]]
-The slope and intercept must be initialized by the user. In addition the learning rate is how quickly to change the parameters by each iteration,this is called alpha in the code. the data set is fitted multiple times with updated starting weights, these are epochs.
+The slope and intercept must be initialized by the user. In addition the learning rate is how quickly to change the parameters by each iteration,this is called alpha in the code. the data set is fit multiple times with updated starting weights, these are epochs.
@@ Line 42: / Line 42: @@
 </code>
-each pair is looped through and a y is predicted according to <math>Ypred=X*m+b</math> The error is then found according to <math>Ypred-Y</math> The weights are then updated according to the formula <math>m:=m-learningrate*error*x</math> and <math>b:=b-learningrate*error</math> The new value of the paramaters are used with the next set of points.
+each pair is looped through and a output is predicted according to <math>Ypred=X*m+b</math> The error is then found according to <math>Ypred-Y</math> The weights are then updated according to the formula <math>m:=m-learningrate*error*x</math> and <math>b:=b-learningrate*error</math> The new value of the paramaters are used with the next set of points.
 <code>
@@ Line 61: / Line 61: @@
 the slope is found to be 1.04 and the y-intercept as -0.08. Imperfect data was used so the exact value was not found. The choice of alpha is important because a value that is too large will give an erroneous result. The algorithm will overshoot the exact value and then update it by overshooting it again resulting in the answer revolving around the exact value.
-This is a graph of the final parameters:
+This is a graph of the final parameters,an appropriate answer was found:
 [[File:Linearregression2tut.png]]

@@ Line 22: / Line 22: @@
 [[File:Scatterlinearpost.png]]
-The slope and intercept must be initialized by the user. In addition the learning rate is how quickly to change the parameters by each iteration. the data set is fitted multiple times with updated starting weights, these are epochs.
+The slope and intercept must be initialized by the user. In addition the learning rate is how quickly to change the parameters by each iteration,this is called alpha in the code. the data set is fitted multiple times with updated starting weights, these are epochs.
@@ Line 42: / Line 42: @@
 </code>
-each pair is looped through and a y is predicted according to <math>Ypred=X*m+b</math> The error is then found according to <math>Ypred-Y</math> The weights are then updated according to the formula <math>m:=m-learning_rate*error*x</math> and <math>b:=b-learning_rate*error</math> The new value of the paramaters are used with the next set of points.
+each pair is looped through and a y is predicted according to <math>Ypred=X*m+b</math> The error is then found according to <math>Ypred-Y</math> The weights are then updated according to the formula <math>m:=m-learningrate*error*x</math> and <math>b:=b-learningrate*error</math> The new value of the paramaters are used with the next set of points.
 <code>

← Older revision		Revision as of 08:06, 12 November 2016
Line 14:		Line 14:


		+
		+
		+	plt.scatter([pair[0][0],pair[1][0],pair[2][0]],[pair[0][1],pair[1][1],pair[2][1]])

	</code>		</code>

@@ Line 6: / Line 6: @@
 <code>
 import numpy as np
 import matplotlib.pyplot as plt
 pair=np.array([(1,1),(4,4.2),(2,2)])
 </code>
@@ Line 19: / Line 25: @@
 nepoch = 4
 m=0.0
 b=0.0
 alpha = 0.2
 </code>
-each pair is looped through and a y is predicted according to <math>Y_pred=X*m+b</math> The error is then found according to <math>Y_pred-Y</math> The weights are then updated according to the formula <math>m:=m-learning_rate*error*x</math> and <math>b:=b-learning_rate*error</math> The new value of the paramaters are used with the next set of points.
+each pair is looped through and a y is predicted according to <math>Ypred=X*m+b</math> The error is then found according to <math>Ypred-Y</math> The weights are then updated according to the formula <math>m:=m-learning_rate*error*x</math> and <math>b:=b-learning_rate*error</math> The new value of the paramaters are used with the next set of points.
 <code>

@@ Line 2: / Line 2: @@
 Here are the packages that are used and three points to approximate a function from.
 <code>
 import numpy as np
@@ Line 12: / Line 14: @@
 The slope and intercept must be initialized by the user. In addition the learning rate is how quickly to change the parameters by each iteration. the data set is fitted multiple times with updated starting weights, these are epochs.
 <code>
@@ Line 36: / Line 39: @@
          m=m-alpha*error*x
          b=b-alpha*error
      print(m,b)
 </code>
-the slope is found to be 1.04 and the y-intercept as -0.08. The exact values were not found because this algorithm approximates the local minimum. It overshoots the exact value and then updates it by overshooting it again resulting in the answer revolving around the exact value.
+the slope is found to be 1.04 and the y-intercept as -0.08. Imperfect data was used so the exact value was not found. The choice of alpha is important because a value that is too large will give an erroneous result. The algorithm will overshoot the exact value and then update it by overshooting it again resulting in the answer revolving around the exact value.