The reason for the differences thus arising from different combinations of equations is easy to find. We have seen that the quantities 1 in equations (1) and (2) are known from observa tion, and therefore subject to the usual small unavoidable observational errors. If the is were absolutely correct, all combinations of equations would give precisely the same results for the three unknowns. But the l's are not thus absolutely correct; and, in a very strict sense, the several equations numbered (2) are therefore inconsistent amongst themselves. Ad mitting fallible numerical values of the l's as actually observed, all equations of the form (2) cannot be exactly true simultaneously: indeed, it follows from the very nature of observational errors such as affect the l's that in all probabil ity no one of the equations (2) is really ab solutely correct and true.
The right-hand members of equations (2), then, Mould not really be zeros. It will be more correct to represent them by a series of small quantities v, and to write equations (2) thus: l.+ (4— te)x+11-= th, (3) 10+ 4+ ( tr—t.) • etc. etc.
Formation of Normal Equations.— It will be convenient at this point to introduce a change of notation, so as to secure greater generality, and at the same time avoid our somewhat cumbersome form, which has been employed simply to visualize our conceptions by fixing attention upon an example derived from actual practice. We shall now increase the number of unknown quantities from two to three, and designate them by the lettetrs x, y, z; and the known coefficients of x, y, a we shall designate by the letters a, b, c. The quantities 1, observa tionally known, will now be designated by Ws. This will bring our notation into conformity with, the usual practice of writers on Least Squares. Our general equation intended for solution thus become, from (3) : (4) aix+biy+ cis+ thx+biy+ cis+ ne-- six+ big etc., etc.
The v's are here simply the errors of the equations, if we may be permitted the use of such an expression. More accurately, they are the amounts by which the equations fail of sat isfaction when we substitute in them any as sumed system of numerical values for the un knowns x, y, s. If, by any process whatever, we obtain a system of numerical values for x, y, a, and substitute those values in the left hand members of equations (4), we shall ob tain a series of residual errors represented by the v's. The name °residual° has therefore been given to the v's, for brevity.
It is evident that if we consider two differ ent systems of numerical values for, the un knowns x, y, a, and substitute both systems in equations (4), we shall get two different sets of values for the residuals v. In fact, and in general, for every different system of values belonging to x, y, a there will be a different set of v's: it is the province of the Method of Least Squares to determine which system of x, y, a, and corresponding v's is preferable. To do this, we make use of the following prin ciple, due to Legendre: Of all possible systems of values for the unknowns x, y, a, that one possesses the highest probability of being correct which makes the sum of the squares of the residuals v a mini mum.
This principle probably does not admit of rigorous demonstration, but it is so highly plausible that we are justified in adopting it without hesitation. This plausibility becomes evident when we consider that the best system of values for x, y, a must certainly make all the individual residuals small. Yet it would not do to adopt as our principle that the sum or mean of the residuals should be a minimum. For some being positive, and some negative, it might happen that the sum or mean would be made very small, while the individual v's re mained in part very large. But the squares are not subject to this criticism, since they are all positive, and are at the same time the simplest functions that are thus positive.
Adopting, then, Legendre's principle of Least Squares, it remains to show how we can use it to solve equations (4). Again following the notation usual with writers on this subject, we shall employ square brackets as a symbol of summation, so that, for instance, the quantity [vv] will designate the sum of the squares of all the v's. Then, according to Legendre's principle, we must have: (5) [vv] minimum.
But, according to equations (4), [vv] is a function of x, y, a; and as these latter quanti ties are independent, the principles of maxima and minima in the calculus tell us that we can satisfy the condition (5) by equating separately to zero the first partial differential coefficients of Ivy] with respect to x, y, a. In other words, we have: Nvl Idyl 6 frei (6) co, 6 = co, ds Now since: • 10ro] = thvt vor. viv4 etc.
equations (6) become: at% , iSt% ek. 0, (7) tAl Va ay dy dv, v, 75;v, 7; + eic. O.