[{"data":1,"prerenderedAt":6809},["ShallowReactive",2],{"article-id-en-ml-basic-2":3},{"id":4,"title":5,"body":6,"description":6407,"extension":6793,"meta":6794,"navigation":6803,"path":31,"seo":6804,"stem":6807,"__hash__":6808},"content/en/blog/ml-basic-2.mdx","Ml Basic 2",{"type":7,"value":8,"toc":6783},"minimark",[9,6396],[10,11,12,16,33,36,41,111,114,117,147,252,349,399,459,528,531,584,620,624,627,655,748,784,787,826,889,955,976,1042,1048,1055,1082,1086,1089,1151,1220,1226,1229,1246,1333,1350,1474,1477,1589,1592,1722,1810,1813,1893,1909,1983,2045,2111,2128,2264,2267,2399,2402,2474,2544,2547,2701,2704,2778,2868,2900,2917,2987,3071,3074,3162,3278,3281,3395,3515,3522,3676,3762,3765,3792,3796,3799,3863,3960,4109,4126,4240,4279,4295,4348,4351,4379,4382,4386,4389,4456,4459,4571,4657,4660,4663,4667,4670,4701,4707,4710,4713,4716,4719,4723,4726,4729,4844,4865,5001,5004,5186,5189,5417,5420,5604,5626,5728,5762,5812,5815,5818,6050,6053,6167,6170,6280],"section-md",{},[13,14,15],"p",{},"This article is part of a series on the fundamentals of machine learning.",[17,18,19,27],"ul",{},[20,21,22],"li",{},[23,24,26],"a",{"href":25},"/en/blog/ml-basic-1","Part 1. About Machine Learning in Simple Terms",[20,28,29],{},[23,30,32],{"href":31},"/en/blog/ml-basic-2","Part 2. Linear Regression as Simple as It Gets",[13,34,35],{},"In the previous article, we looked at machine learning in general,\nwithout going into the details of how it works. In this article, we'll\nstart exploring specific algorithms. And we'll begin with what I consider\nthe simplest model — linear regression.",[37,38,40],"h2",{"id":39},"understanding-the-terminology","Understanding the Terminology",[13,42,43,44,67,68,72,73,86,87,110],{},"First, let's figure out what problem we're solving and how. So, we're\nsolving a regression problem. We discussed what this kind of problem is\nin the previous article, but just in case, here's a reminder. We have\nsome data ",[45,46,49],"span",{"className":47},[48],"katex",[50,51,53],"math",{"xmlns":52},"http://www.w3.org/1998/Math/MathML",[54,55,56,63],"semantics",{},[57,58,59],"mrow",{},[60,61,62],"mi",{},"X",[64,65,62],"annotation",{"encoding":66},"application/x-tex",". It can be a single number or several (such a set of\nnumbers is called a ",[69,70,71],"strong",{},"vector","). We want to obtain a single value for ",[45,74,76],{"className":75},[48],[50,77,78],{"xmlns":52},[54,79,80,84],{},[57,81,82],{},[60,83,62],{},[64,85,62],{"encoding":66},"\nas the output, which we denote as ",[45,88,90],{"className":89},[48],[50,91,92],{"xmlns":52},[54,93,94,107],{},[57,95,96],{},[97,98,100,103],"mover",{"accent":99},"true",[60,101,102],{},"y",[104,105,106],"mo",{"stretchy":99},"^",[64,108,109],{"encoding":66},"\\widehat{y}",". This value is\ncontinuous and can be in any range.",[13,112,113],{},"As an example of such a problem, consider determining a person's weight\nbased on their height. Then our input data is a single number — the\nperson's height in centimeters. The output is also a single number —\nweight in kilograms. How do we calculate this value?",[13,115,116],{},"The second word in the method's name gives us a hint. Our regression is\nlinear. What does this mean? It means that to determine the output value,\nwe'll use the formula of a straight line. Here it's worth recalling the\nschool curriculum, which states that a straight line is defined by the\nfollowing expression:",[45,118,120],{"className":119},[48],[50,121,123],{"xmlns":52,"display":122},"block",[54,124,125,144],{},[57,126,127,129,132,135,138,141],{},[60,128,102],{},[104,130,131],{},"=",[60,133,134],{},"k",[60,136,137],{},"x",[104,139,140],{},"+",[60,142,143],{},"b",[64,145,146],{"encoding":66},"y = kx + b",[13,148,149,150,163,164,177,178,191,192,205,206,223,224,237,238,251],{},"This formula is easy to adapt for our case. Simply denote ",[45,151,153],{"className":152},[48],[50,154,155],{"xmlns":52},[54,156,157,161],{},[57,158,159],{},[60,160,102],{},[64,162,102],{"encoding":66}," as the\nperson's weight, and ",[45,165,167],{"className":166},[48],[50,168,169],{"xmlns":52},[54,170,171,175],{},[57,172,173],{},[60,174,137],{},[64,176,137],{"encoding":66}," as their height. Actually, we need to slightly\nadjust the notation. Usually, ",[45,179,181],{"className":180},[48],[50,182,183],{"xmlns":52},[54,184,185,189],{},[57,186,187],{},[60,188,102],{},[64,190,102],{"encoding":66}," denotes the correct answer, that is,\nin our case, the person's actual weight. We, however, are trying to make\na prediction, and our result, as mentioned above, will be denoted not as\n",[45,193,195],{"className":194},[48],[50,196,197],{"xmlns":52},[54,198,199,203],{},[57,200,201],{},[60,202,102],{},[64,204,102],{"encoding":66},", but as ",[45,207,209],{"className":208},[48],[50,210,211],{"xmlns":52},[54,212,213,221],{},[57,214,215],{},[97,216,217,219],{"accent":99},[60,218,102],{},[104,220,106],{"stretchy":99},[64,222,109],{"encoding":66},". But let's return from notation to the essence.\nAll we need to do is find the coefficients ",[45,225,227],{"className":226},[48],[50,228,229],{"xmlns":52},[54,230,231,235],{},[57,232,233],{},[60,234,134],{},[64,236,134],{"encoding":66}," and ",[45,239,241],{"className":240},[48],[50,242,243],{"xmlns":52},[54,244,245,249],{},[57,246,247],{},[60,248,143],{},[64,250,143],{"encoding":66},", and we can\napproximately solve the problem.",[13,253,254,255,268,269,310,311,329,330,348],{},"But the solution won't be very accurate (and most likely — very\ninaccurate). The thing is, height and weight are not directly related by\na linear dependency. There are tall and thin people, and there are short\nand heavy people. Such a simple formula will work poorly. To increase\naccuracy, we need to make the model more complex. Let's add waist\ncircumference. Now our input data is no longer a single number ",[45,256,258],{"className":257},[48],[50,259,260],{"xmlns":52},[54,261,262,266],{},[57,263,264],{},[60,265,137],{},[64,267,137],{"encoding":66},", but a\nvector ",[45,270,272],{"className":271},[48],[50,273,274],{"xmlns":52},[54,275,276,307],{},[57,277,278,281,290,293,297,304],{},[104,279,280],{"fence":99},"[",[282,283,284,286],"msub",{},[60,285,137],{},[287,288,289],"mn",{},"1",[104,291,292],{"separator":99},",",[294,295,296],"mtext",{}," ",[282,298,299,301],{},[60,300,137],{},[287,302,303],{},"2",[104,305,306],{"fence":99},"]",[64,308,309],{"encoding":66},"\\left\\lbrack x_{1},\\ x_{2} \\right\\rbrack",": ",[45,312,314],{"className":313},[48],[50,315,316],{"xmlns":52},[54,317,318,326],{},[57,319,320],{},[282,321,322,324],{},[60,323,137],{},[287,325,289],{},[64,327,328],{"encoding":66},"x_{1}"," denotes\nheight, and ",[45,331,333],{"className":332},[48],[50,334,335],{"xmlns":52},[54,336,337,345],{},[57,338,339],{},[282,340,341,343],{},[60,342,137],{},[287,344,303],{},[64,346,347],{"encoding":66},"x_{2}"," — waist circumference. Our output data remains the\nsame — weight in kilograms. The formula changes, but not radically:",[45,350,352],{"className":351},[48],[50,353,354],{"xmlns":52,"display":122},[54,355,356,396],{},[57,357,358,364,366,372,378,380,386,392,394],{},[97,359,360,362],{"accent":99},[60,361,102],{},[104,363,106],{"stretchy":99},[104,365,131],{},[282,367,368,370],{},[60,369,134],{},[287,371,289],{},[282,373,374,376],{},[60,375,137],{},[287,377,289],{},[104,379,140],{},[282,381,382,384],{},[60,383,134],{},[287,385,303],{},[282,387,388,390],{},[60,389,137],{},[287,391,303],{},[104,393,140],{},[60,395,143],{},[64,397,398],{"encoding":66},"\\widehat{y} = k_{1}x_{1} + k_{2}x_{2} + b",[13,400,401,402,237,430,443,444,458],{},"For successful training, we now need to find not two, but three\nparameters: ",[45,403,405],{"className":404},[48],[50,406,407],{"xmlns":52},[54,408,409,427],{},[57,410,411,417,419,421],{},[282,412,413,415],{},[60,414,134],{},[287,416,289],{},[104,418,292],{"separator":99},[294,420,296],{},[282,422,423,425],{},[60,424,134],{},[287,426,303],{},[64,428,429],{"encoding":66},"k_{1},\\ k_{2}",[45,431,433],{"className":432},[48],[50,434,435],{"xmlns":52},[54,436,437,441],{},[57,438,439],{},[60,440,143],{},[64,442,143],{"encoding":66},". To further improve accuracy, we can\nkeep adding input data, for example, leg length, chest circumference, and\nso on. Moreover, each new number in the input data brings with it a new\nmodel parameter that we'll need to find. In general form, if we have many\nparameters (let's denote this \"many\" as ",[45,445,447],{"className":446},[48],[50,448,449],{"xmlns":52},[54,450,451,456],{},[57,452,453],{},[60,454,455],{},"m",[64,457,455],{"encoding":66},"), the formula will look like\nthis:",[45,460,462],{"className":461},[48],[50,463,464],{"xmlns":52,"display":122},[54,465,466,525],{},[57,467,468,474,476,482,488,490,496,502,504,507,509,515,521,523],{},[97,469,470,472],{"accent":99},[60,471,102],{},[104,473,106],{"stretchy":99},[104,475,131],{},[282,477,478,480],{},[60,479,134],{},[287,481,289],{},[282,483,484,486],{},[60,485,137],{},[287,487,289],{},[104,489,140],{},[282,491,492,494],{},[60,493,134],{},[287,495,303],{},[282,497,498,500],{},[60,499,137],{},[287,501,303],{},[104,503,140],{},[104,505,506],{},"…",[104,508,140],{},[282,510,511,513],{},[60,512,134],{},[60,514,455],{},[282,516,517,519],{},[60,518,137],{},[60,520,455],{},[104,522,140],{},[60,524,143],{},[64,526,527],{"encoding":66},"\\widehat{y} = k_{1}x_{1} + k_{2}x_{2} + \\ldots + k_{m}x_{m} + b",[13,529,530],{},"or",[45,532,534],{"className":533},[48],[50,535,536],{"xmlns":52,"display":122},[54,537,538,581],{},[57,539,540,546,548,565,571,577,579],{},[97,541,542,544],{"accent":99},[60,543,102],{},[104,545,106],{"stretchy":99},[104,547,131],{},[549,550,551,554,563],"munderover",{},[104,552,553],{},"∑",[57,555,556,559,561],{},[60,557,558],{},"i",[104,560,131],{},[287,562,289],{},[60,564,455],{},[282,566,567,569],{},[60,568,134],{},[60,570,558],{},[282,572,573,575],{},[60,574,137],{},[60,576,558],{},[104,578,140],{},[60,580,143],{},[64,582,583],{"encoding":66},"\\widehat{y} = \\sum_{i = 1}^{m} k_{i}x_{i}  + b",[13,585,586,587,605,606,619],{},"All that's left is to figure out how to find the parameters ",[45,588,590],{"className":589},[48],[50,591,592],{"xmlns":52},[54,593,594,602],{},[57,595,596],{},[282,597,598,600],{},[60,599,134],{},[60,601,558],{},[64,603,604],{"encoding":66},"k_{i}"," and\n",[45,607,609],{"className":608},[48],[50,610,611],{"xmlns":52},[54,612,613,617],{},[57,614,615],{},[60,616,143],{},[64,618,143],{"encoding":66},".",[37,621,623],{"id":622},"learning-to-learn","Learning to Learn",[13,625,626],{},"First, let's return to the simplest case where we have one parameter. In\nthis form, our formula looks like this:",[45,628,630],{"className":629},[48],[50,631,632],{"xmlns":52,"display":122},[54,633,634,652],{},[57,635,636,642,644,646,648,650],{},[97,637,638,640],{"accent":99},[60,639,102],{},[104,641,106],{"stretchy":99},[104,643,131],{},[60,645,134],{},[60,647,137],{},[104,649,140],{},[60,651,143],{},[64,653,654],{"encoding":66},"\\widehat{y} = kx + b",[13,656,657,658,605,671,684,685,702,703,716,717,605,730,747],{},"As mentioned above, training consists of finding the coefficients ",[45,659,661],{"className":660},[48],[50,662,663],{"xmlns":52},[54,664,665,669],{},[57,666,667],{},[60,668,134],{},[64,670,134],{"encoding":66},[45,672,674],{"className":673},[48],[50,675,676],{"xmlns":52},[54,677,678,682],{},[57,679,680],{},[60,681,143],{},[64,683,143],{"encoding":66},". What should these coefficients be? We want our answers\n",[45,686,688],{"className":687},[48],[50,689,690],{"xmlns":52},[54,691,692,700],{},[57,693,694],{},[97,695,696,698],{"accent":99},[60,697,102],{},[104,699,106],{"stretchy":99},[64,701,109],{"encoding":66}," to be as close as possible to the correct answers ",[45,704,706],{"className":705},[48],[50,707,708],{"xmlns":52},[54,709,710,714],{},[57,711,712],{},[60,713,102],{},[64,715,102],{"encoding":66},".\nWhat does \"as close as possible\" mean? The difference between ",[45,718,720],{"className":719},[48],[50,721,722],{"xmlns":52},[54,723,724,728],{},[57,725,726],{},[60,727,102],{},[64,729,102],{"encoding":66},[45,731,733],{"className":732},[48],[50,734,735],{"xmlns":52},[54,736,737,745],{},[57,738,739],{},[97,740,741,743],{"accent":99},[60,742,102],{},[104,744,106],{"stretchy":99},[64,746,109],{"encoding":66}," should be minimal. This can be written as",[45,749,751],{"className":750},[48],[50,752,753],{"xmlns":52,"display":122},[54,754,755,781],{},[57,756,757,775,778],{},[57,758,759,762,764,767,773],{},[104,760,761],{"fence":99},"∣",[60,763,102],{},[104,765,766],{},"−",[97,768,769,771],{"accent":99},[60,770,102],{},[104,772,106],{"stretchy":99},[104,774,761],{"fence":99},[104,776,777],{},"→",[287,779,780],{},"0",[64,782,783],{"encoding":66},"\\left| y - \\widehat{y} \\right| \\rightarrow 0",[13,785,786],{},"Working with absolute values is inconvenient, so it's better to use the\nsquare instead. Mathematically, this is equivalent (if the absolute value\ntends to zero, then the square tends to zero as well):",[45,788,790],{"className":789},[48],[50,791,792],{"xmlns":52,"display":122},[54,793,794,823],{},[57,795,796,819,821],{},[797,798,799,817],"msup",{},[57,800,801,804,806,808,814],{},[104,802,803],{"fence":99},"(",[60,805,102],{},[104,807,766],{},[97,809,810,812],{"accent":99},[60,811,102],{},[104,813,106],{"stretchy":99},[104,815,816],{"fence":99},")",[287,818,303],{},[104,820,777],{},[287,822,780],{},[64,824,825],{"encoding":66},"\\left( y - \\widehat{y} \\right)^{2} \\rightarrow 0",[13,827,828,829,237,842,855,856,883,884,888],{},"If we had only one example, the problem would be trivial: choose any pair\nof ",[45,830,832],{"className":831},[48],[50,833,834],{"xmlns":52},[54,835,836,840],{},[57,837,838],{},[60,839,134],{},[64,841,134],{"encoding":66},[45,843,845],{"className":844},[48],[50,846,847],{"xmlns":52},[54,848,849,853],{},[57,850,851],{},[60,852,143],{},[64,854,143],{"encoding":66}," such that ",[45,857,859],{"className":858},[48],[50,860,861],{"xmlns":52},[54,862,863,880],{},[57,864,865,867,870,872,874,876,878],{},[60,866,134],{},[104,868,869],{},"∗",[60,871,137],{},[104,873,140],{},[60,875,143],{},[104,877,131],{},[60,879,102],{},[64,881,882],{"encoding":66},"k*x + b = y",". There are infinitely many such\npairs, which follows from geometric considerations — infinitely many\nlines can be drawn through a single point. But we want our model to work\nnot for just one pair of values, but for all possible ones. That is, we\nneed to draw the line so that it is ",[885,886,887],"em",{},"on average"," as close as possible to\nall our points:",[45,890,892],{"className":891},[48],[50,893,894],{"xmlns":52,"display":122},[54,895,896,952],{},[57,897,898,948,950],{},[899,900,901,946],"mfrac",{},[57,902,903,918],{},[549,904,905,907,915],{},[104,906,553],{},[57,908,909,911,913],{},[60,910,558],{},[104,912,131],{},[287,914,289],{},[60,916,917],{},"n",[797,919,920,944],{},[57,921,922,924,930,932,942],{},[104,923,803],{"fence":99},[282,925,926,928],{},[60,927,102],{},[60,929,558],{},[104,931,766],{},[97,933,934,940],{"accent":99},[282,935,936,938],{},[60,937,102],{},[60,939,558],{},[104,941,106],{"stretchy":99},[104,943,816],{"fence":99},[287,945,303],{},[60,947,917],{},[104,949,777],{},[287,951,780],{},[64,953,954],{"encoding":66},"\\frac{ \\sum_{i = 1}^{n} \\left( y_{i} - \\widehat{y_{i}} \\right)^{2} }{n} \\rightarrow 0",[13,956,957,958,975],{},"We can ignore the denominator since it doesn't affect anything. Now let's\nrecall how ",[45,959,961],{"className":960},[48],[50,962,963],{"xmlns":52},[54,964,965,973],{},[57,966,967],{},[97,968,969,971],{"accent":99},[60,970,102],{},[104,972,106],{"stretchy":99},[64,974,109],{"encoding":66}," is calculated:",[45,977,979],{"className":978},[48],[50,980,981],{"xmlns":52,"display":122},[54,982,983,1039],{},[57,984,985,999,1035,1037],{},[549,986,987,989,997],{},[104,988,553],{},[57,990,991,993,995],{},[60,992,558],{},[104,994,131],{},[287,996,289],{},[60,998,917],{},[797,1000,1001,1033],{},[57,1002,1003,1005,1011,1013,1031],{},[104,1004,803],{"fence":99},[282,1006,1007,1009],{},[60,1008,102],{},[60,1010,558],{},[104,1012,766],{},[57,1014,1015,1017,1019,1025,1027,1029],{},[104,1016,803],{"fence":99},[60,1018,134],{},[282,1020,1021,1023],{},[60,1022,137],{},[60,1024,558],{},[104,1026,140],{},[60,1028,143],{},[104,1030,816],{"fence":99},[104,1032,816],{"fence":99},[287,1034,303],{},[104,1036,777],{},[287,1038,780],{},[64,1040,1041],{"encoding":66},"\\sum_{i = 1}^{n}\\left( y_{i} - \\left( kx_{i} + b \\right) \\right)^{2} \\rightarrow 0",[13,1043,1044,1045,619],{},"Optimizing such a function is called the ",[69,1046,1047],{},"least squares method",[13,1049,1050],{},[1051,1052],"img",{"alt":1053,"src":1054},"Illustration of the least squares method","/img/blog/ml-basic-2/image1.png",[13,1056,1057,1058,1081],{},"In the illustration, blue dots represent the actual values. Red dots —\nthe model's predictions. The red line — the graph of the function\n",[45,1059,1061],{"className":1060},[48],[50,1062,1063],{"xmlns":52},[54,1064,1065,1079],{},[57,1066,1067,1069,1071,1073,1075,1077],{},[60,1068,102],{},[104,1070,131],{},[60,1072,134],{},[60,1074,137],{},[104,1076,140],{},[60,1078,143],{},[64,1080,146],{"encoding":66},". We try to minimize the sum of the distances between the\nred and blue dots (dashed lines).",[37,1083,1085],{"id":1084},"optimizing","Optimizing",[13,1087,1088],{},"Let's look at our function:",[45,1090,1092],{"className":1091},[48],[50,1093,1094],{"xmlns":52,"display":122},[54,1095,1096,1148],{},[57,1097,1098,1112],{},[549,1099,1100,1102,1110],{},[104,1101,553],{},[57,1103,1104,1106,1108],{},[60,1105,558],{},[104,1107,131],{},[287,1109,289],{},[60,1111,917],{},[797,1113,1114,1146],{},[57,1115,1116,1118,1124,1126,1144],{},[104,1117,803],{"fence":99},[282,1119,1120,1122],{},[60,1121,102],{},[60,1123,558],{},[104,1125,766],{},[57,1127,1128,1130,1132,1138,1140,1142],{},[104,1129,803],{"fence":99},[60,1131,134],{},[282,1133,1134,1136],{},[60,1135,137],{},[60,1137,558],{},[104,1139,140],{},[60,1141,143],{},[104,1143,816],{"fence":99},[104,1145,816],{"fence":99},[287,1147,303],{},[64,1149,1150],{"encoding":66},"\\sum_{i = 1}^{n}\\left( y_{i} - \\left( kx_{i} + b \\right) \\right)^{2}",[13,1152,1153,1154,1172,1173,1191,1192,1205,1206,1219],{},"What does it depend on? The input data ",[45,1155,1157],{"className":1156},[48],[50,1158,1159],{"xmlns":52},[54,1160,1161,1169],{},[57,1162,1163],{},[282,1164,1165,1167],{},[60,1166,137],{},[60,1168,558],{},[64,1170,1171],{"encoding":66},"x_{i}"," and the correct answers\n",[45,1174,1176],{"className":1175},[48],[50,1177,1178],{"xmlns":52},[54,1179,1180,1188],{},[57,1181,1182],{},[282,1183,1184,1186],{},[60,1185,102],{},[60,1187,558],{},[64,1189,1190],{"encoding":66},"y_{i}"," are fixed. Therefore, the sum depends only on our parameters ",[45,1193,1195],{"className":1194},[48],[50,1196,1197],{"xmlns":52},[54,1198,1199,1203],{},[57,1200,1201],{},[60,1202,134],{},[64,1204,134],{"encoding":66},"\nand ",[45,1207,1209],{"className":1208},[48],[50,1210,1211],{"xmlns":52},[54,1212,1213,1217],{},[57,1214,1215],{},[60,1216,143],{},[64,1218,143],{"encoding":66},". It turns out we have a function of two variables, and we need to\nfind its minimum. If we draw the surface described by this function, we\nget the following picture:",[13,1221,1222],{},[1051,1223],{"alt":1224,"src":1225},"Surface described by a quadratic function","/img/blog/ml-basic-2/image2.png",[13,1227,1228],{},"If we recall school mathematics again, the extremum (minimum or maximum)\nof a function is found where the derivative equals zero (if such a point\nexists for the function). This is exactly our case: from the graph, we see\nthat there is only one extremum, and it will be the minimum.",[13,1230,1231,1232,1245],{},"The most important thing is that we can compute derivatives separately\nfor each of our variables. First, we compute the derivative with respect\nto ",[45,1233,1235],{"className":1234},[48],[50,1236,1237],{"xmlns":52},[54,1238,1239,1243],{},[57,1240,1241],{},[60,1242,143],{},[64,1244,143],{"encoding":66},". Our sum is simply an addition operation. Let's recall the chain\nrule:",[45,1247,1249],{"className":1248},[48],[50,1250,1251],{"xmlns":52,"display":122},[54,1252,1253,1330],{},[57,1254,1255,1266,1269,1272,1275,1277,1279,1281,1283,1285,1295,1297,1299,1301,1303,1305,1307,1309,1312,1322,1324,1326,1328],{},[899,1256,1257,1260],{},[60,1258,1259],{},"d",[57,1261,1262,1264],{},[60,1263,1259],{},[60,1265,137],{},[60,1267,1268],{},"f",[104,1270,803],{"stretchy":1271},"false",[60,1273,1274],{},"g",[104,1276,803],{"stretchy":1271},[60,1278,137],{},[104,1280,816],{"stretchy":1271},[104,1282,816],{"stretchy":1271},[104,1284,131],{},[899,1286,1287,1289],{},[60,1288,1259],{},[57,1290,1291,1293],{},[60,1292,1259],{},[60,1294,1274],{},[60,1296,1268],{},[104,1298,803],{"stretchy":1271},[60,1300,1274],{},[104,1302,803],{"stretchy":1271},[60,1304,137],{},[104,1306,816],{"stretchy":1271},[104,1308,816],{"stretchy":1271},[104,1310,1311],{},"⋅",[899,1313,1314,1316],{},[60,1315,1259],{},[57,1317,1318,1320],{},[60,1319,1259],{},[60,1321,137],{},[60,1323,1274],{},[104,1325,803],{"stretchy":1271},[60,1327,137],{},[104,1329,816],{"stretchy":1271},[64,1331,1332],{"encoding":66},"\\frac{d}{dx} f(g(x)) = \\frac{d}{dg} f(g(x)) \\cdot \\frac{d}{dx} g(x)",[13,1334,1335,1336,1349],{},"First, we compute with respect to ",[45,1337,1339],{"className":1338},[48],[50,1340,1341],{"xmlns":52},[54,1342,1343,1347],{},[57,1344,1345],{},[60,1346,143],{},[64,1348,143],{"encoding":66},":",[45,1351,1353],{"className":1352},[48],[50,1354,1355],{"xmlns":52,"display":122},[54,1356,1357,1471],{},[57,1358,1359,1369,1383,1419,1421,1423,1425,1439],{},[899,1360,1361,1363],{},[60,1362,1259],{},[57,1364,1365,1367],{},[60,1366,1259],{},[60,1368,143],{},[549,1370,1371,1373,1381],{},[104,1372,553],{},[57,1374,1375,1377,1379],{},[60,1376,558],{},[104,1378,131],{},[287,1380,289],{},[60,1382,917],{},[797,1384,1385,1417],{},[57,1386,1387,1389,1395,1397,1415],{},[104,1388,803],{"fence":99},[282,1390,1391,1393],{},[60,1392,102],{},[60,1394,558],{},[104,1396,766],{},[57,1398,1399,1401,1403,1409,1411,1413],{},[104,1400,803],{"fence":99},[60,1402,134],{},[282,1404,1405,1407],{},[60,1406,137],{},[60,1408,558],{},[104,1410,140],{},[60,1412,143],{},[104,1414,816],{"fence":99},[104,1416,816],{"fence":99},[287,1418,303],{},[104,1420,131],{},[104,1422,766],{},[287,1424,303],{},[549,1426,1427,1429,1437],{},[104,1428,553],{},[57,1430,1431,1433,1435],{},[60,1432,558],{},[104,1434,131],{},[287,1436,289],{},[60,1438,917],{},[57,1440,1441,1443,1449,1451,1469],{},[104,1442,803],{"fence":99},[282,1444,1445,1447],{},[60,1446,102],{},[60,1448,558],{},[104,1450,766],{},[57,1452,1453,1455,1457,1463,1465,1467],{},[104,1454,803],{"fence":99},[60,1456,134],{},[282,1458,1459,1461],{},[60,1460,137],{},[60,1462,558],{},[104,1464,140],{},[60,1466,143],{},[104,1468,816],{"fence":99},[104,1470,816],{"fence":99},[64,1472,1473],{"encoding":66},"\\frac{d}{db}\\sum_{i = 1}^{n}\\left( y_{i} - \\left( kx_{i} + b \\right) \\right)^{2} = - 2\\sum_{i = 1}^{n}\\left( y_{i} - \\left( kx_{i} + b \\right) \\right)",[13,1475,1476],{},"Let's remove the minus in front of the sum by simply swapping the\nexpressions in parentheses:",[45,1478,1480],{"className":1479},[48],[50,1481,1482],{"xmlns":52,"display":122},[54,1483,1484,1586],{},[57,1485,1486,1488,1490,1504,1536,1538,1540,1554],{},[104,1487,766],{},[287,1489,303],{},[549,1491,1492,1494,1502],{},[104,1493,553],{},[57,1495,1496,1498,1500],{},[60,1497,558],{},[104,1499,131],{},[287,1501,289],{},[60,1503,917],{},[57,1505,1506,1508,1514,1516,1534],{},[104,1507,803],{"fence":99},[282,1509,1510,1512],{},[60,1511,102],{},[60,1513,558],{},[104,1515,766],{},[57,1517,1518,1520,1522,1528,1530,1532],{},[104,1519,803],{"fence":99},[60,1521,134],{},[282,1523,1524,1526],{},[60,1525,137],{},[60,1527,558],{},[104,1529,140],{},[60,1531,143],{},[104,1533,816],{"fence":99},[104,1535,816],{"fence":99},[104,1537,131],{},[287,1539,303],{},[549,1541,1542,1544,1552],{},[104,1543,553],{},[57,1545,1546,1548,1550],{},[60,1547,558],{},[104,1549,131],{},[287,1551,289],{},[60,1553,917],{},[57,1555,1556,1558,1576,1578,1584],{},[104,1557,803],{"fence":99},[57,1559,1560,1562,1564,1570,1572,1574],{},[104,1561,803],{"fence":99},[60,1563,134],{},[282,1565,1566,1568],{},[60,1567,137],{},[60,1569,558],{},[104,1571,140],{},[60,1573,143],{},[104,1575,816],{"fence":99},[104,1577,766],{},[282,1579,1580,1582],{},[60,1581,102],{},[60,1583,558],{},[104,1585,816],{"fence":99},[64,1587,1588],{"encoding":66},"-2\\sum_{i = 1}^{n}\\left( y_{i} - \\left( kx_{i} + b \\right) \\right) = 2\\sum_{i = 1}^{n}\\left( \\left( kx_{i} + b \\right) - y_{i} \\right)",[13,1590,1591],{},"Since we no longer have a square, we can expand the expression into\nseveral sums:",[45,1593,1595],{"className":1594},[48],[50,1596,1597],{"xmlns":52,"display":122},[54,1598,1599,1719],{},[57,1600,1601,1603,1617,1649,1651,1653,1667,1669,1675,1677,1679,1693,1695,1697,1699,1713],{},[287,1602,303],{},[549,1604,1605,1607,1615],{},[104,1606,553],{},[57,1608,1609,1611,1613],{},[60,1610,558],{},[104,1612,131],{},[287,1614,289],{},[60,1616,917],{},[57,1618,1619,1621,1639,1641,1647],{},[104,1620,803],{"fence":99},[57,1622,1623,1625,1627,1633,1635,1637],{},[104,1624,803],{"fence":99},[60,1626,134],{},[282,1628,1629,1631],{},[60,1630,137],{},[60,1632,558],{},[104,1634,140],{},[60,1636,143],{},[104,1638,816],{"fence":99},[104,1640,766],{},[282,1642,1643,1645],{},[60,1644,102],{},[60,1646,558],{},[104,1648,816],{"fence":99},[104,1650,131],{},[287,1652,303],{},[549,1654,1655,1657,1665],{},[104,1656,553],{},[57,1658,1659,1661,1663],{},[60,1660,558],{},[104,1662,131],{},[287,1664,289],{},[60,1666,917],{},[60,1668,134],{},[282,1670,1671,1673],{},[60,1672,137],{},[60,1674,558],{},[104,1676,140],{},[287,1678,303],{},[549,1680,1681,1683,1691],{},[104,1682,553],{},[57,1684,1685,1687,1689],{},[60,1686,558],{},[104,1688,131],{},[287,1690,289],{},[60,1692,917],{},[60,1694,143],{},[104,1696,766],{},[287,1698,303],{},[549,1700,1701,1703,1711],{},[104,1702,553],{},[57,1704,1705,1707,1709],{},[60,1706,558],{},[104,1708,131],{},[287,1710,289],{},[60,1712,917],{},[282,1714,1715,1717],{},[60,1716,102],{},[60,1718,558],{},[64,1720,1721],{"encoding":66},"2\\sum_{i = 1}^{n}\\left( \\left( kx_{i} + b \\right) - y_{i} \\right) = 2\\sum_{i = 1}^{n}kx_{i} + 2\\sum_{i = 1}^{n} b-2 \\sum_{i = 1}^{n} y_{i}",[13,1723,1724,1725,1756,1757,1770,1771,1787,1788,619],{},"Note that ",[45,1726,1728],{"className":1727},[48],[50,1729,1730],{"xmlns":52},[54,1731,1732,1753],{},[57,1733,1734,1736,1751],{},[287,1735,303],{},[1737,1738,1739,1741,1749],"msubsup",{},[104,1740,553],{},[57,1742,1743,1745,1747],{},[60,1744,558],{},[104,1746,131],{},[287,1748,289],{},[60,1750,917],{},[60,1752,143],{},[64,1754,1755],{"encoding":66},"2\\sum_{i = 1}^{n}b"," is simply adding ",[45,1758,1760],{"className":1759},[48],[50,1761,1762],{"xmlns":52},[54,1763,1764,1768],{},[57,1765,1766],{},[60,1767,143],{},[64,1769,143],{"encoding":66}," to itself ",[45,1772,1774],{"className":1773},[48],[50,1775,1776],{"xmlns":52},[54,1777,1778,1784],{},[57,1779,1780,1782],{},[287,1781,303],{},[60,1783,917],{},[64,1785,1786],{"encoding":66},"2n","\ntimes, that is, ",[45,1789,1791],{"className":1790},[48],[50,1792,1793],{"xmlns":52},[54,1794,1795,1807],{},[57,1796,1797,1799,1801,1803,1805],{},[287,1798,303],{},[104,1800,869],{},[60,1802,917],{},[104,1804,869],{},[60,1806,143],{},[64,1808,1809],{"encoding":66},"2*n*b",[13,1811,1812],{},"Now recall that we're looking for where the derivative equals zero. So\nlet's set our expression equal to zero:",[45,1814,1816],{"className":1815},[48],[50,1817,1818],{"xmlns":52,"display":122},[54,1819,1820,1890],{},[57,1821,1822,1824,1838,1852,1854,1856,1858,1860,1862,1864,1878],{},[287,1823,303],{},[549,1825,1826,1828,1836],{},[104,1827,553],{},[57,1829,1830,1832,1834],{},[60,1831,558],{},[104,1833,131],{},[287,1835,289],{},[60,1837,917],{},[57,1839,1840,1842,1844,1850],{},[104,1841,803],{"fence":99},[60,1843,134],{},[282,1845,1846,1848],{},[60,1847,137],{},[60,1849,558],{},[104,1851,816],{"fence":99},[104,1853,140],{},[287,1855,303],{},[60,1857,917],{},[60,1859,143],{},[104,1861,766],{},[287,1863,303],{},[549,1865,1866,1868,1876],{},[104,1867,553],{},[57,1869,1870,1872,1874],{},[60,1871,558],{},[104,1873,131],{},[287,1875,289],{},[60,1877,917],{},[57,1879,1880,1886,1888],{},[282,1881,1882,1884],{},[60,1883,102],{},[60,1885,558],{},[104,1887,131],{},[287,1889,780],{},[64,1891,1892],{"encoding":66},"2\\sum_{i = 1}^{n}\\left( kx_{i} \\right) + 2nb - 2\\sum_{i = 1}^{n}{y_{i} = 0}",[13,1894,1895,1896,1349],{},"Cancel out the twos and rearrange the terms to express ",[45,1897,1899],{"className":1898},[48],[50,1900,1901],{"xmlns":52},[54,1902,1903,1907],{},[57,1904,1905],{},[60,1906,143],{},[64,1908,143],{"encoding":66},[45,1910,1912],{"className":1911},[48],[50,1913,1914],{"xmlns":52,"display":122},[54,1915,1916,1980],{},[57,1917,1918,1932,1946,1948,1950,1952,1954,1968],{},[549,1919,1920,1922,1930],{},[104,1921,553],{},[57,1923,1924,1926,1928],{},[60,1925,558],{},[104,1927,131],{},[287,1929,289],{},[60,1931,917],{},[57,1933,1934,1936,1938,1944],{},[104,1935,803],{"fence":99},[60,1937,134],{},[282,1939,1940,1942],{},[60,1941,137],{},[60,1943,558],{},[104,1945,816],{"fence":99},[104,1947,140],{},[60,1949,917],{},[60,1951,143],{},[104,1953,766],{},[549,1955,1956,1958,1966],{},[104,1957,553],{},[57,1959,1960,1962,1964],{},[60,1961,558],{},[104,1963,131],{},[287,1965,289],{},[60,1967,917],{},[57,1969,1970,1976,1978],{},[282,1971,1972,1974],{},[60,1973,102],{},[60,1975,558],{},[104,1977,131],{},[287,1979,780],{},[64,1981,1982],{"encoding":66},"\\sum_{i = 1}^{n}\\left( kx_{i} \\right) + nb - \\sum_{i = 1}^{n}{y_{i} = 0}",[45,1984,1986],{"className":1985},[48],[50,1987,1988],{"xmlns":52,"display":122},[54,1989,1990,2042],{},[57,1991,1992,1994,1996,1998,2012,2018,2020,2034,2036],{},[60,1993,917],{},[60,1995,143],{},[104,1997,131],{},[549,1999,2000,2002,2010],{},[104,2001,553],{},[57,2003,2004,2006,2008],{},[60,2005,558],{},[104,2007,131],{},[287,2009,289],{},[60,2011,917],{},[282,2013,2014,2016],{},[60,2015,102],{},[60,2017,558],{},[104,2019,766],{},[549,2021,2022,2024,2032],{},[104,2023,553],{},[57,2025,2026,2028,2030],{},[60,2027,558],{},[104,2029,131],{},[287,2031,289],{},[60,2033,917],{},[60,2035,134],{},[282,2037,2038,2040],{},[60,2039,137],{},[60,2041,558],{},[64,2043,2044],{"encoding":66},"nb = \\sum_{i = 1}^{n} y_{i} - \\sum_{i = 1}^{n} kx_{i}",[45,2046,2048],{"className":2047},[48],[50,2049,2050],{"xmlns":52,"display":122},[54,2051,2052,2108],{},[57,2053,2054,2056,2058],{},[60,2055,143],{},[104,2057,131],{},[899,2059,2060,2106],{},[57,2061,2062,2076,2082,2084,2086,2100],{},[549,2063,2064,2066,2074],{},[104,2065,553],{},[57,2067,2068,2070,2072],{},[60,2069,558],{},[104,2071,131],{},[287,2073,289],{},[60,2075,917],{},[282,2077,2078,2080],{},[60,2079,102],{},[60,2081,558],{},[104,2083,766],{},[60,2085,134],{},[549,2087,2088,2090,2098],{},[104,2089,553],{},[57,2091,2092,2094,2096],{},[60,2093,558],{},[104,2095,131],{},[287,2097,289],{},[60,2099,917],{},[282,2101,2102,2104],{},[60,2103,137],{},[60,2105,558],{},[60,2107,917],{},[64,2109,2110],{"encoding":66},"b = \\frac{\\sum_{i = 1}^{n} y_{i} - k\\sum_{i = 1}^{n} x_{i} }{n}",[13,2112,2113,2114,2127],{},"Now let's compute the derivative with respect to ",[45,2115,2117],{"className":2116},[48],[50,2118,2119],{"xmlns":52},[54,2120,2121,2125],{},[57,2122,2123],{},[60,2124,134],{},[64,2126,134],{"encoding":66}," in the same way:",[45,2129,2131],{"className":2130},[48],[50,2132,2133],{"xmlns":52,"display":122},[54,2134,2135,2261],{},[57,2136,2137,2147,2161,2197,2199,2201,2203,2217],{},[899,2138,2139,2141],{},[60,2140,1259],{},[57,2142,2143,2145],{},[60,2144,1259],{},[60,2146,134],{},[549,2148,2149,2151,2159],{},[104,2150,553],{},[57,2152,2153,2155,2157],{},[60,2154,558],{},[104,2156,131],{},[287,2158,289],{},[60,2160,917],{},[797,2162,2163,2195],{},[57,2164,2165,2167,2173,2175,2193],{},[104,2166,803],{"fence":99},[282,2168,2169,2171],{},[60,2170,102],{},[60,2172,558],{},[104,2174,766],{},[57,2176,2177,2179,2181,2187,2189,2191],{},[104,2178,803],{"fence":99},[60,2180,134],{},[282,2182,2183,2185],{},[60,2184,137],{},[60,2186,558],{},[104,2188,140],{},[60,2190,143],{},[104,2192,816],{"fence":99},[104,2194,816],{"fence":99},[287,2196,303],{},[104,2198,131],{},[104,2200,766],{},[287,2202,303],{},[549,2204,2205,2207,2215],{},[104,2206,553],{},[57,2208,2209,2211,2213],{},[60,2210,558],{},[104,2212,131],{},[287,2214,289],{},[60,2216,917],{},[57,2218,2219,2221,2253,2259],{},[104,2220,803],{"fence":99},[57,2222,2223,2225,2231,2233,2251],{},[104,2224,803],{"fence":99},[282,2226,2227,2229],{},[60,2228,102],{},[60,2230,558],{},[104,2232,766],{},[57,2234,2235,2237,2239,2245,2247,2249],{},[104,2236,803],{"fence":99},[60,2238,134],{},[282,2240,2241,2243],{},[60,2242,137],{},[60,2244,558],{},[104,2246,140],{},[60,2248,143],{},[104,2250,816],{"fence":99},[104,2252,816],{"fence":99},[282,2254,2255,2257],{},[60,2256,137],{},[60,2258,558],{},[104,2260,816],{"fence":99},[64,2262,2263],{"encoding":66},"\\frac{d}{dk} \\sum_{i = 1}^{n} \\left( y_{i} - \\left( kx_{i} + b \\right) \\right)^{2} =  - 2\\sum_{i = 1}^{n}\\left( \\left( y_{i} - \\left( kx_{i} + b \\right) \\right)x_{i} \\right)",[13,2265,2266],{},"Rearrange the terms to remove the minus:",[45,2268,2270],{"className":2269},[48],[50,2271,2272],{"xmlns":52,"display":122},[54,2273,2274,2396],{},[57,2275,2276,2278,2280,2294,2340,2342,2344,2358],{},[104,2277,766],{},[287,2279,303],{},[549,2281,2282,2284,2292],{},[104,2283,553],{},[57,2285,2286,2288,2290],{},[60,2287,558],{},[104,2289,131],{},[287,2291,289],{},[60,2293,917],{},[57,2295,2296,2298,2330,2332,2338],{},[104,2297,803],{"fence":99},[57,2299,2300,2302,2308,2310,2328],{},[104,2301,803],{"fence":99},[282,2303,2304,2306],{},[60,2305,102],{},[60,2307,558],{},[104,2309,766],{},[57,2311,2312,2314,2316,2322,2324,2326],{},[104,2313,803],{"fence":99},[60,2315,134],{},[282,2317,2318,2320],{},[60,2319,137],{},[60,2321,558],{},[104,2323,140],{},[60,2325,143],{},[104,2327,816],{"fence":99},[104,2329,816],{"fence":99},[104,2331,1311],{},[282,2333,2334,2336],{},[60,2335,137],{},[60,2337,558],{},[104,2339,816],{"fence":99},[104,2341,131],{},[287,2343,303],{},[549,2345,2346,2348,2356],{},[104,2347,553],{},[57,2349,2350,2352,2354],{},[60,2351,558],{},[104,2353,131],{},[287,2355,289],{},[60,2357,917],{},[57,2359,2360,2362,2388,2394],{},[104,2361,803],{"fence":99},[57,2363,2364,2366,2368,2374,2376,2378,2380,2386],{},[104,2365,803],{"fence":99},[60,2367,134],{},[282,2369,2370,2372],{},[60,2371,137],{},[60,2373,558],{},[104,2375,140],{},[60,2377,143],{},[104,2379,766],{},[282,2381,2382,2384],{},[60,2383,102],{},[60,2385,558],{},[104,2387,816],{"fence":99},[282,2389,2390,2392],{},[60,2391,137],{},[60,2393,558],{},[104,2395,816],{"fence":99},[64,2397,2398],{"encoding":66},"-2\\sum_{i = 1}^{n}\\left( \\left( y_{i} - \\left( kx_{i} + b \\right) \\right) \\cdot x_{i} \\right) = 2\\sum_{i = 1}^{n}{\\left( \\left( kx_{i} + b - y_{i} \\right)x_{i} \\right)}",[13,2400,2401],{},"Set it equal to zero and cancel the two:",[45,2403,2405],{"className":2404},[48],[50,2406,2407],{"xmlns":52,"display":122},[54,2408,2409,2471],{},[57,2410,2411,2413,2427,2467,2469],{},[287,2412,303],{},[549,2414,2415,2417,2425],{},[104,2416,553],{},[57,2418,2419,2421,2423],{},[60,2420,558],{},[104,2422,131],{},[287,2424,289],{},[60,2426,917],{},[57,2428,2429,2431,2457,2459,2465],{},[104,2430,803],{"fence":99},[57,2432,2433,2435,2437,2443,2445,2447,2449,2455],{},[104,2434,803],{"fence":99},[60,2436,134],{},[282,2438,2439,2441],{},[60,2440,137],{},[60,2442,558],{},[104,2444,140],{},[60,2446,143],{},[104,2448,766],{},[282,2450,2451,2453],{},[60,2452,102],{},[60,2454,558],{},[104,2456,816],{"fence":99},[104,2458,1311],{},[282,2460,2461,2463],{},[60,2462,137],{},[60,2464,558],{},[104,2466,816],{"fence":99},[104,2468,131],{},[287,2470,780],{},[64,2472,2473],{"encoding":66},"2\\sum_{i = 1}^{n} \\left( \\left( kx_{i} + b - y_{i} \\right) \\cdot x_{i} \\right) = 0",[45,2475,2477],{"className":2476},[48],[50,2478,2479],{"xmlns":52,"display":122},[54,2480,2481,2541],{},[57,2482,2483,2497,2537,2539],{},[549,2484,2485,2487,2495],{},[104,2486,553],{},[57,2488,2489,2491,2493],{},[60,2490,558],{},[104,2492,131],{},[287,2494,289],{},[60,2496,917],{},[57,2498,2499,2501,2527,2529,2535],{},[104,2500,803],{"fence":99},[57,2502,2503,2505,2507,2513,2515,2517,2519,2525],{},[104,2504,803],{"fence":99},[60,2506,134],{},[282,2508,2509,2511],{},[60,2510,137],{},[60,2512,558],{},[104,2514,140],{},[60,2516,143],{},[104,2518,766],{},[282,2520,2521,2523],{},[60,2522,102],{},[60,2524,558],{},[104,2526,816],{"fence":99},[104,2528,1311],{},[282,2530,2531,2533],{},[60,2532,137],{},[60,2534,558],{},[104,2536,816],{"fence":99},[104,2538,131],{},[287,2540,780],{},[64,2542,2543],{"encoding":66},"\\sum_{i = 1}^{n}\\left( \\left( kx_{i} + b - y_{i} \\right) \\cdot x_{i} \\right) = 0",[13,2545,2546],{},"As a result, we get a system of two equations:",[45,2548,2550],{"className":2549},[48],[50,2551,2552],{"xmlns":52,"display":122},[54,2553,2554,2698],{},[57,2555,2556,2559],{},[104,2557,2558],{"fence":99},"{",[2560,2561,2565,2636],"mtable",{"rowspacing":2562,"columnalign":2563,"columnspacing":2564},"0.36em","left left","1em",[2566,2567,2568],"mtr",{},[2569,2570,2571],"mtd",{},[2572,2573,2574],"mstyle",{"scriptlevel":780,"displaystyle":1271},[57,2575,2576,2590,2632,2634],{},[1737,2577,2578,2580,2588],{},[104,2579,553],{},[57,2581,2582,2584,2586],{},[60,2583,558],{},[104,2585,131],{},[287,2587,289],{},[60,2589,917],{},[57,2591,2592,2594,2622,2624,2630],{},[104,2593,803],{"fence":99},[57,2595,2596,2598,2600,2602,2608,2610,2612,2614,2620],{},[104,2597,803],{"fence":99},[60,2599,134],{},[104,2601,869],{},[282,2603,2604,2606],{},[60,2605,137],{},[60,2607,558],{},[104,2609,140],{},[60,2611,143],{},[104,2613,766],{},[282,2615,2616,2618],{},[60,2617,102],{},[60,2619,558],{},[104,2621,816],{"fence":99},[104,2623,1311],{},[282,2625,2626,2628],{},[60,2627,137],{},[60,2629,558],{},[104,2631,816],{"fence":99},[104,2633,131],{},[287,2635,780],{},[2566,2637,2638],{},[2569,2639,2640],{},[2572,2641,2642],{"scriptlevel":780,"displaystyle":1271},[57,2643,2644,2646,2648],{},[60,2645,143],{},[104,2647,131],{},[899,2649,2650,2696],{},[57,2651,2652,2666,2672,2674,2676,2690],{},[1737,2653,2654,2656,2664],{},[104,2655,553],{},[57,2657,2658,2660,2662],{},[60,2659,558],{},[104,2661,131],{},[287,2663,289],{},[60,2665,917],{},[282,2667,2668,2670],{},[60,2669,102],{},[60,2671,558],{},[104,2673,766],{},[60,2675,134],{},[1737,2677,2678,2680,2688],{},[104,2679,553],{},[57,2681,2682,2684,2686],{},[60,2683,558],{},[104,2685,131],{},[287,2687,289],{},[60,2689,917],{},[282,2691,2692,2694],{},[60,2693,137],{},[60,2695,558],{},[60,2697,917],{},[64,2699,2700],{"encoding":66},"\\begin{cases}\n\\sum_{i = 1}^{n} \\left( \\left(k * x_{i} + b - y_{i}\\right) \\cdot x_{i} \\right) = 0 \\\\\nb = \\frac{ \\sum_{i = 1}^{n} y_{i} - k \\sum_{i = 1}^{n} x_{i} }{n}\n\\end{cases}",[13,2702,2703],{},"This system can be simplified a bit. Let's look at the second equation.\nLet's represent it as:",[45,2705,2707],{"className":2706},[48],[50,2708,2709],{"xmlns":52,"display":122},[54,2710,2711,2775],{},[57,2712,2713,2715,2717,2743,2745,2747,2749],{},[60,2714,143],{},[104,2716,131],{},[899,2718,2719,2741],{},[57,2720,2721,2735],{},[549,2722,2723,2725,2733],{},[104,2724,553],{},[57,2726,2727,2729,2731],{},[60,2728,558],{},[104,2730,131],{},[287,2732,289],{},[60,2734,917],{},[282,2736,2737,2739],{},[60,2738,102],{},[60,2740,558],{},[60,2742,917],{},[104,2744,766],{},[60,2746,134],{},[104,2748,1311],{},[899,2750,2751,2773],{},[57,2752,2753,2767],{},[549,2754,2755,2757,2765],{},[104,2756,553],{},[57,2758,2759,2761,2763],{},[60,2760,558],{},[104,2762,131],{},[287,2764,289],{},[60,2766,917],{},[282,2768,2769,2771],{},[60,2770,137],{},[60,2772,558],{},[60,2774,917],{},[64,2776,2777],{"encoding":66},"b = \\frac{ \\sum_{i = 1}^{n} y_{i} }{n} - k \\sdot \\frac{ \\sum_{i = 1}^{n} x_{i} }{n}",[13,2779,2780,2781,2819,2820,2833,2834,2853,2854,2867],{},"What is ",[45,2782,2784],{"className":2783},[48],[50,2785,2786],{"xmlns":52},[54,2787,2788,2816],{},[57,2789,2790],{},[899,2791,2792,2814],{},[57,2793,2794,2808],{},[1737,2795,2796,2798,2806],{},[104,2797,553],{},[57,2799,2800,2802,2804],{},[60,2801,558],{},[104,2803,131],{},[287,2805,289],{},[60,2807,917],{},[282,2809,2810,2812],{},[60,2811,102],{},[60,2813,558],{},[60,2815,917],{},[64,2817,2818],{"encoding":66},"\\frac{\\sum_{i = 1}^{n}y_{i}}{n}","? It's the mean value of ",[45,2821,2823],{"className":2822},[48],[50,2824,2825],{"xmlns":52},[54,2826,2827,2831],{},[57,2828,2829],{},[60,2830,102],{},[64,2832,102],{"encoding":66},"\n(because we're dividing the sum of all elements by their count). Let's\ndenote it as ",[45,2835,2837],{"className":2836},[48],[50,2838,2839],{"xmlns":52},[54,2840,2841,2850],{},[57,2842,2843],{},[97,2844,2845,2847],{"accent":99},[60,2846,102],{},[104,2848,2849],{"stretchy":99},"‾",[64,2851,2852],{"encoding":66},"\\overline{y}",". We'll do the same for ",[45,2855,2857],{"className":2856},[48],[50,2858,2859],{"xmlns":52},[54,2860,2861,2865],{},[57,2862,2863],{},[60,2864,137],{},[64,2866,137],{"encoding":66},". As a result, we\nget:",[45,2869,2871],{"className":2870},[48],[50,2872,2873],{"xmlns":52,"display":122},[54,2874,2875,2897],{},[57,2876,2877,2879,2881,2887,2889,2891],{},[60,2878,143],{},[104,2880,131],{},[97,2882,2883,2885],{"accent":99},[60,2884,102],{},[104,2886,2849],{"stretchy":99},[104,2888,766],{},[60,2890,134],{},[97,2892,2893,2895],{"accent":99},[60,2894,137],{},[104,2896,2849],{"stretchy":99},[64,2898,2899],{"encoding":66},"b = \\overline{y} - k  \\overline{x}",[13,2901,2902,2903,2916],{},"Agree: without all those sums, it's much simpler. Now let's do what we\ndid in school: substitute ",[45,2904,2906],{"className":2905},[48],[50,2907,2908],{"xmlns":52},[54,2909,2910,2914],{},[57,2911,2912],{},[60,2913,143],{},[64,2915,143],{"encoding":66}," into the first equation.",[45,2918,2920],{"className":2919},[48],[50,2921,2922],{"xmlns":52,"display":122},[54,2923,2924,2984],{},[57,2925,2926,2940,2980,2982],{},[549,2927,2928,2930,2938],{},[104,2929,553],{},[57,2931,2932,2934,2936],{},[60,2933,558],{},[104,2935,131],{},[287,2937,289],{},[60,2939,917],{},[57,2941,2942,2944,2970,2972,2978],{},[104,2943,803],{"fence":99},[57,2945,2946,2948,2950,2956,2958,2960,2962,2968],{},[104,2947,803],{"fence":99},[60,2949,134],{},[282,2951,2952,2954],{},[60,2953,137],{},[60,2955,558],{},[104,2957,140],{},[60,2959,143],{},[104,2961,766],{},[282,2963,2964,2966],{},[60,2965,102],{},[60,2967,558],{},[104,2969,816],{"fence":99},[104,2971,1311],{},[282,2973,2974,2976],{},[60,2975,137],{},[60,2977,558],{},[104,2979,816],{"fence":99},[104,2981,131],{},[287,2983,780],{},[64,2985,2986],{"encoding":66},"\\sum_{i = 1}^{n} \\left( \\left( kx_{i} + b - y_{i} \\right) \\cdot x_{i} \\right) = 0",[45,2988,2990],{"className":2989},[48],[50,2991,2992],{"xmlns":52,"display":122},[54,2993,2994,3068],{},[57,2995,2996,3010,3064,3066],{},[549,2997,2998,3000,3008],{},[104,2999,553],{},[57,3001,3002,3004,3006],{},[60,3003,558],{},[104,3005,131],{},[287,3007,289],{},[60,3009,917],{},[57,3011,3012,3014,3054,3056,3062],{},[104,3013,803],{"fence":99},[57,3015,3016,3018,3020,3026,3028,3034,3036,3038,3044,3046,3052],{},[104,3017,803],{"fence":99},[60,3019,134],{},[282,3021,3022,3024],{},[60,3023,137],{},[60,3025,558],{},[104,3027,140],{},[97,3029,3030,3032],{"accent":99},[60,3031,102],{},[104,3033,2849],{"stretchy":99},[104,3035,766],{},[60,3037,134],{},[97,3039,3040,3042],{"accent":99},[60,3041,137],{},[104,3043,2849],{"stretchy":99},[104,3045,766],{},[282,3047,3048,3050],{},[60,3049,102],{},[60,3051,558],{},[104,3053,816],{"fence":99},[104,3055,1311],{},[282,3057,3058,3060],{},[60,3059,137],{},[60,3061,558],{},[104,3063,816],{"fence":99},[104,3065,131],{},[287,3067,780],{},[64,3069,3070],{"encoding":66},"\\sum_{i = 1}^{n} \\left( \\left( kx_{i} + \\overline{y} - k\\overline{x} - y_{i} \\right) \\cdot x_{i} \\right) = 0",[13,3072,3073],{},"Let's simplify everything we can and expand the brackets:",[45,3075,3077],{"className":3076},[48],[50,3078,3079],{"xmlns":52,"display":122},[54,3080,3081,3159],{},[57,3082,3083,3097,3155,3157],{},[549,3084,3085,3087,3095],{},[104,3086,553],{},[57,3088,3089,3091,3093],{},[60,3090,558],{},[104,3092,131],{},[287,3094,289],{},[60,3096,917],{},[57,3098,3099,3101,3145,3147,3153],{},[104,3100,803],{"fence":99},[57,3102,3103,3105,3107,3117,3119,3125,3127,3129,3135,3137,3143],{},[104,3104,803],{"fence":99},[60,3106,134],{},[282,3108,3109,3115],{},[57,3110,3111,3113],{},[104,3112,803],{"stretchy":1271},[60,3114,137],{},[60,3116,558],{},[104,3118,766],{},[97,3120,3121,3123],{"accent":99},[60,3122,137],{},[104,3124,2849],{"stretchy":99},[104,3126,816],{"stretchy":1271},[104,3128,140],{},[97,3130,3131,3133],{"accent":99},[60,3132,102],{},[104,3134,2849],{"stretchy":99},[104,3136,766],{},[282,3138,3139,3141],{},[60,3140,102],{},[60,3142,558],{},[104,3144,816],{"fence":99},[104,3146,1311],{},[282,3148,3149,3151],{},[60,3150,137],{},[60,3152,558],{},[104,3154,816],{"fence":99},[104,3156,131],{},[287,3158,780],{},[64,3160,3161],{"encoding":66},"\\sum_{i = 1}^{n} \\left( \\left( k{(x}_{i} - \\overline{x}) + \\overline{y} - y_{i} \\right) \\cdot x_{i} \\right) = 0",[45,3163,3165],{"className":3164},[48],[50,3166,3167],{"xmlns":52,"display":122},[54,3168,3169,3275],{},[57,3170,3171,3173,3187,3221,3223,3237,3271,3273],{},[60,3172,134],{},[549,3174,3175,3177,3185],{},[104,3176,553],{},[57,3178,3179,3181,3183],{},[60,3180,558],{},[104,3182,131],{},[287,3184,289],{},[60,3186,917],{},[57,3188,3189,3191,3211,3213,3219],{},[104,3190,803],{"fence":99},[57,3192,3193,3195,3201,3203,3209],{},[104,3194,803],{"fence":99},[282,3196,3197,3199],{},[60,3198,137],{},[60,3200,558],{},[104,3202,766],{},[97,3204,3205,3207],{"accent":99},[60,3206,137],{},[104,3208,2849],{"stretchy":99},[104,3210,816],{"fence":99},[104,3212,1311],{},[282,3214,3215,3217],{},[60,3216,137],{},[60,3218,558],{},[104,3220,816],{"fence":99},[104,3222,140],{},[549,3224,3225,3227,3235],{},[104,3226,553],{},[57,3228,3229,3231,3233],{},[60,3230,558],{},[104,3232,131],{},[287,3234,289],{},[60,3236,917],{},[57,3238,3239,3241,3261,3263,3269],{},[104,3240,803],{"fence":99},[57,3242,3243,3245,3251,3253,3259],{},[104,3244,803],{"fence":99},[97,3246,3247,3249],{"accent":99},[60,3248,102],{},[104,3250,2849],{"stretchy":99},[104,3252,766],{},[282,3254,3255,3257],{},[60,3256,102],{},[60,3258,558],{},[104,3260,816],{"fence":99},[104,3262,1311],{},[282,3264,3265,3267],{},[60,3266,137],{},[60,3268,558],{},[104,3270,816],{"fence":99},[104,3272,131],{},[287,3274,780],{},[64,3276,3277],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( \\left( x_{i} - \\overline{x} \\right) \\cdot x_{i} \\right) +\n\\sum_{i = 1}^{n} \\left( \\left( \\overline{y} - y_{i} \\right) \\cdot x_{i} \\right) = 0",[13,3279,3280],{},"In principle, this is already enough to calculate everything. We can\nsimply move the second term to the other side and divide:",[45,3282,3284],{"className":3283},[48],[50,3285,3286],{"xmlns":52,"display":122},[54,3287,3288,3392],{},[57,3289,3290,3292,3306,3340,3342,3344,3358],{},[60,3291,134],{},[549,3293,3294,3296,3304],{},[104,3295,553],{},[57,3297,3298,3300,3302],{},[60,3299,558],{},[104,3301,131],{},[287,3303,289],{},[60,3305,917],{},[57,3307,3308,3310,3330,3332,3338],{},[104,3309,803],{"fence":99},[57,3311,3312,3314,3320,3322,3328],{},[104,3313,803],{"fence":99},[282,3315,3316,3318],{},[60,3317,137],{},[60,3319,558],{},[104,3321,766],{},[97,3323,3324,3326],{"accent":99},[60,3325,137],{},[104,3327,2849],{"stretchy":99},[104,3329,816],{"fence":99},[104,3331,1311],{},[282,3333,3334,3336],{},[60,3335,137],{},[60,3337,558],{},[104,3339,816],{"fence":99},[104,3341,131],{},[104,3343,766],{},[549,3345,3346,3348,3356],{},[104,3347,553],{},[57,3349,3350,3352,3354],{},[60,3351,558],{},[104,3353,131],{},[287,3355,289],{},[60,3357,917],{},[57,3359,3360,3362,3382,3384,3390],{},[104,3361,803],{"fence":99},[57,3363,3364,3366,3372,3374,3380],{},[104,3365,803],{"fence":99},[97,3367,3368,3370],{"accent":99},[60,3369,102],{},[104,3371,2849],{"stretchy":99},[104,3373,766],{},[282,3375,3376,3378],{},[60,3377,102],{},[60,3379,558],{},[104,3381,816],{"fence":99},[104,3383,1311],{},[282,3385,3386,3388],{},[60,3387,137],{},[60,3389,558],{},[104,3391,816],{"fence":99},[64,3393,3394],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( \\left( x_{i} - \\overline{x} \\right) \\cdot x_{i} \\right) =\n- \\sum_{i = 1}^{n} \\left( \\left( \\overline{y} - y_{i} \\right) \\cdot x_{i} \\right)",[45,3396,3398],{"className":3397},[48],[50,3399,3400],{"xmlns":52,"display":122},[54,3401,3402,3512],{},[57,3403,3404,3406,3408,3410],{},[60,3405,134],{},[104,3407,131],{},[104,3409,766],{},[899,3411,3412,3462],{},[57,3413,3414,3428],{},[549,3415,3416,3418,3426],{},[104,3417,553],{},[57,3419,3420,3422,3424],{},[60,3421,558],{},[104,3423,131],{},[287,3425,289],{},[60,3427,917],{},[57,3429,3430,3432,3452,3454,3460],{},[104,3431,803],{"fence":99},[57,3433,3434,3436,3442,3444,3450],{},[104,3435,803],{"fence":99},[97,3437,3438,3440],{"accent":99},[60,3439,102],{},[104,3441,2849],{"stretchy":99},[104,3443,766],{},[282,3445,3446,3448],{},[60,3447,102],{},[60,3449,558],{},[104,3451,816],{"fence":99},[104,3453,1311],{},[282,3455,3456,3458],{},[60,3457,137],{},[60,3459,558],{},[104,3461,816],{"fence":99},[57,3463,3464,3478],{},[549,3465,3466,3468,3476],{},[104,3467,553],{},[57,3469,3470,3472,3474],{},[60,3471,558],{},[104,3473,131],{},[287,3475,289],{},[60,3477,917],{},[57,3479,3480,3482,3502,3504,3510],{},[104,3481,803],{"fence":99},[57,3483,3484,3486,3492,3494,3500],{},[104,3485,803],{"fence":99},[282,3487,3488,3490],{},[60,3489,137],{},[60,3491,558],{},[104,3493,766],{},[97,3495,3496,3498],{"accent":99},[60,3497,137],{},[104,3499,2849],{"stretchy":99},[104,3501,816],{"fence":99},[104,3503,1311],{},[282,3505,3506,3508],{},[60,3507,137],{},[60,3509,558],{},[104,3511,816],{"fence":99},[64,3513,3514],{"encoding":66},"k = - \\frac{\n  \\sum_{i = 1}^{n} \\left( \\left( \\overline{y} - y_{i} \\right) \\sdot x_{i} \\right)\n}{\n  \\sum_{i = 1}^{n} \\left( \\left( x_{i} - \\overline{x} \\right) \\cdot x_{i} \\right)\n}",[13,3516,3517,3518,3521],{},"The formula can be simplified further. ",[69,3519,3520],{},"I'll show exactly how at the end\nof the article, so as not to add even more formulas here",".\nAs a result, we get:",[45,3523,3525],{"className":3524},[48],[50,3526,3527],{"xmlns":52,"display":122},[54,3528,3529,3673],{},[57,3530,3531,3533],{},[104,3532,2558],{"fence":99},[2560,3534,3535,3645],{"rowspacing":2562,"columnalign":2563,"columnspacing":2564},[2566,3536,3537],{},[2569,3538,3539],{},[2572,3540,3541],{"scriptlevel":780,"displaystyle":1271},[57,3542,3543,3545,3547],{},[60,3544,134],{},[104,3546,131],{},[899,3548,3549,3605],{},[57,3550,3551,3565,3585],{},[1737,3552,3553,3555,3563],{},[104,3554,553],{},[57,3556,3557,3559,3561],{},[60,3558,558],{},[104,3560,131],{},[287,3562,289],{},[60,3564,917],{},[57,3566,3567,3569,3575,3577,3583],{},[104,3568,803],{"fence":99},[282,3570,3571,3573],{},[60,3572,137],{},[60,3574,558],{},[104,3576,766],{},[97,3578,3579,3581],{"accent":99},[60,3580,137],{},[104,3582,2849],{"stretchy":99},[104,3584,816],{"fence":99},[57,3586,3587,3589,3595,3597,3603],{},[104,3588,803],{"fence":99},[282,3590,3591,3593],{},[60,3592,102],{},[60,3594,558],{},[104,3596,766],{},[97,3598,3599,3601],{"accent":99},[60,3600,102],{},[104,3602,2849],{"stretchy":99},[104,3604,816],{"fence":99},[57,3606,3607,3621],{},[1737,3608,3609,3611,3619],{},[104,3610,553],{},[57,3612,3613,3615,3617],{},[60,3614,558],{},[104,3616,131],{},[287,3618,289],{},[60,3620,917],{},[797,3622,3623,3643],{},[57,3624,3625,3627,3633,3635,3641],{},[104,3626,803],{"fence":99},[282,3628,3629,3631],{},[60,3630,137],{},[60,3632,558],{},[104,3634,766],{},[97,3636,3637,3639],{"accent":99},[60,3638,137],{},[104,3640,2849],{"stretchy":99},[104,3642,816],{"fence":99},[287,3644,303],{},[2566,3646,3647],{},[2569,3648,3649],{},[2572,3650,3651],{"scriptlevel":780,"displaystyle":1271},[57,3652,3653,3655,3657,3663,3665,3667],{},[60,3654,143],{},[104,3656,131],{},[97,3658,3659,3661],{"accent":99},[60,3660,102],{},[104,3662,2849],{"stretchy":99},[104,3664,766],{},[60,3666,134],{},[97,3668,3669,3671],{"accent":99},[60,3670,137],{},[104,3672,2849],{"stretchy":99},[64,3674,3675],{"encoding":66},"\\begin{cases}\nk = \\frac{\n  \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right) \\left(y_{i} - \\overline{y} \\right)\n}{\n  \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right)^{2}\n} \\\\\nb = \\overline{y} - k\\overline{x}\n\\end{cases}",[13,3677,3678,3679,237,3692,3705,3706,3719,3720,3733,3734,3747,3748,3761],{},"Thus, for any set of ",[45,3680,3682],{"className":3681},[48],[50,3683,3684],{"xmlns":52},[54,3685,3686,3690],{},[57,3687,3688],{},[60,3689,137],{},[64,3691,137],{"encoding":66},[45,3693,3695],{"className":3694},[48],[50,3696,3697],{"xmlns":52},[54,3698,3699,3703],{},[57,3700,3701],{},[60,3702,102],{},[64,3704,102],{"encoding":66},", we can calculate our coefficients.\nAfter all, ",[45,3707,3709],{"className":3708},[48],[50,3710,3711],{"xmlns":52},[54,3712,3713,3717],{},[57,3714,3715],{},[60,3716,134],{},[64,3718,134],{"encoding":66}," doesn't depend on ",[45,3721,3723],{"className":3722},[48],[50,3724,3725],{"xmlns":52},[54,3726,3727,3731],{},[57,3728,3729],{},[60,3730,143],{},[64,3732,143],{"encoding":66},". We calculate it using the formula,\nand then, knowing ",[45,3735,3737],{"className":3736},[48],[50,3738,3739],{"xmlns":52},[54,3740,3741,3745],{},[57,3742,3743],{},[60,3744,134],{},[64,3746,134],{"encoding":66},", we find ",[45,3749,3751],{"className":3750},[48],[50,3752,3753],{"xmlns":52},[54,3754,3755,3759],{},[57,3756,3757],{},[60,3758,143],{},[64,3760,143],{"encoding":66},". And now we have our first working\nmachine learning algorithm.",[13,3763,3764],{},"Now that we have all our coefficients, we can easily calculate the answer\nfor any input parameters using the same formula:",[45,3766,3768],{"className":3767},[48],[50,3769,3770],{"xmlns":52,"display":122},[54,3771,3772,3790],{},[57,3773,3774,3780,3782,3784,3786,3788],{},[97,3775,3776,3778],{"accent":99},[60,3777,102],{},[104,3779,106],{"stretchy":99},[104,3781,131],{},[60,3783,134],{},[60,3785,137],{},[104,3787,140],{},[60,3789,143],{},[64,3791,654],{"encoding":66},[37,3793,3795],{"id":3794},"example-usage","Example Usage",[13,3797,3798],{},"Let's return to the problem of determining a person's weight based on their\nheight. Suppose we have the following dataset:",[3800,3801,3802,3819],"table",{},[3803,3804,3805],"thead",{},[3806,3807,3808,3814],"tr",{},[3809,3810,3811],"th",{},[69,3812,3813],{},"Height (x), cm",[3809,3815,3816],{},[69,3817,3818],{},"Weight (y), kg",[3820,3821,3822,3831,3839,3847,3855],"tbody",{},[3806,3823,3824,3828],{},[3825,3826,3827],"td",{},"154",[3825,3829,3830],{},"43",[3806,3832,3833,3836],{},[3825,3834,3835],{},"196",[3825,3837,3838],{},"107",[3806,3840,3841,3844],{},[3825,3842,3843],{},"172",[3825,3845,3846],{},"73",[3806,3848,3849,3852],{},[3825,3850,3851],{},"185",[3825,3853,3854],{},"80",[3806,3856,3857,3860],{},[3825,3858,3859],{},"161",[3825,3861,3862],{},"66",[13,3864,3865,3866,3879,3880,3905,3906,3919,3920,3945,3946,3959],{},"First, let's find the mean values. For ",[45,3867,3869],{"className":3868},[48],[50,3870,3871],{"xmlns":52},[54,3872,3873,3877],{},[57,3874,3875],{},[60,3876,137],{},[64,3878,137],{"encoding":66}," it will be\n",[45,3881,3883],{"className":3882},[48],[50,3884,3885],{"xmlns":52},[54,3886,3887,3902],{},[57,3888,3889,3895,3897,3899],{},[97,3890,3891,3893],{"accent":99},[60,3892,137],{},[104,3894,2849],{"stretchy":99},[104,3896,131],{},[294,3898,296],{},[287,3900,3901],{},"173.6",[64,3903,3904],{"encoding":66},"\\overline{x} = \\ 173.6",", for ",[45,3907,3909],{"className":3908},[48],[50,3910,3911],{"xmlns":52},[54,3912,3913,3917],{},[57,3914,3915],{},[60,3916,102],{},[64,3918,102],{"encoding":66}," — ",[45,3921,3923],{"className":3922},[48],[50,3924,3925],{"xmlns":52},[54,3926,3927,3942],{},[57,3928,3929,3935,3937,3939],{},[97,3930,3931,3933],{"accent":99},[60,3932,102],{},[104,3934,2849],{"stretchy":99},[104,3936,131],{},[294,3938,296],{},[287,3940,3941],{},"73.8",[64,3943,3944],{"encoding":66},"\\overline{y} = \\ 73.8",". Now we need\nto find the coefficients. Let's start with ",[45,3947,3949],{"className":3948},[48],[50,3950,3951],{"xmlns":52},[54,3952,3953,3957],{},[57,3954,3955],{},[60,3956,134],{},[64,3958,134],{"encoding":66},". For each element, we\ncalculate:",[3800,3961,3962,4047],{},[3803,3963,3964],{},[3806,3965,3966,3983,3999,4023],{},[3809,3967,3968],{},[45,3969,3971],{"className":3970},[48],[50,3972,3973],{"xmlns":52},[54,3974,3975,3980],{},[57,3976,3977],{},[60,3978,137],{"mathvariant":3979},"bold",[64,3981,3982],{"encoding":66},"\\mathbf{x}",[3809,3984,3985],{},[45,3986,3988],{"className":3987},[48],[50,3989,3990],{"xmlns":52},[54,3991,3992,3996],{},[57,3993,3994],{},[60,3995,102],{"mathvariant":3979},[64,3997,3998],{"encoding":66},"\\mathbf{y}",[3809,4000,4001],{},[45,4002,4004],{"className":4003},[48],[50,4005,4006],{"xmlns":52},[54,4007,4008,4020],{},[57,4009,4010,4012,4014],{},[60,4011,137],{"mathvariant":3979},[104,4013,766],{},[97,4015,4016,4018],{"accent":99},[60,4017,137],{"mathvariant":3979},[104,4019,2849],{"stretchy":99},[64,4021,4022],{"encoding":66},"\\mathbf{x - \\overline{x}}",[3809,4024,4025],{},[45,4026,4028],{"className":4027},[48],[50,4029,4030],{"xmlns":52},[54,4031,4032,4044],{},[57,4033,4034,4036,4038],{},[60,4035,102],{"mathvariant":3979},[104,4037,766],{},[97,4039,4040,4042],{"accent":99},[60,4041,102],{"mathvariant":3979},[104,4043,2849],{"stretchy":99},[64,4045,4046],{"encoding":66},"\\mathbf{y - \\overline{y}}",[3820,4048,4049,4061,4073,4085,4097],{},[3806,4050,4051,4053,4055,4058],{},[3825,4052,3827],{},[3825,4054,3830],{},[3825,4056,4057],{},"-19.6",[3825,4059,4060],{},"-30.8",[3806,4062,4063,4065,4067,4070],{},[3825,4064,3835],{},[3825,4066,3838],{},[3825,4068,4069],{},"22.4",[3825,4071,4072],{},"33.2",[3806,4074,4075,4077,4079,4082],{},[3825,4076,3843],{},[3825,4078,3846],{},[3825,4080,4081],{},"-1.6",[3825,4083,4084],{},"-0.8",[3806,4086,4087,4089,4091,4094],{},[3825,4088,3851],{},[3825,4090,3854],{},[3825,4092,4093],{},"11.4",[3825,4095,4096],{},"6.2",[3806,4098,4099,4101,4103,4106],{},[3825,4100,3859],{},[3825,4102,3862],{},[3825,4104,4105],{},"-12.6",[3825,4107,4108],{},"-7.8",[13,4110,4111,4112,4125],{},"Now we have everything to calculate ",[45,4113,4115],{"className":4114},[48],[50,4116,4117],{"xmlns":52},[54,4118,4119,4123],{},[57,4120,4121],{},[60,4122,134],{},[64,4124,134],{"encoding":66}," using the formula:",[45,4127,4129],{"className":4128},[48],[50,4130,4131],{"xmlns":52,"display":122},[54,4132,4133,4237],{},[57,4134,4135,4137,4139],{},[60,4136,134],{},[104,4138,131],{},[899,4140,4141,4197],{},[57,4142,4143,4157,4177],{},[549,4144,4145,4147,4155],{},[104,4146,553],{},[57,4148,4149,4151,4153],{},[60,4150,558],{},[104,4152,131],{},[287,4154,289],{},[60,4156,917],{},[57,4158,4159,4161,4167,4169,4175],{},[104,4160,803],{"fence":99},[282,4162,4163,4165],{},[60,4164,137],{},[60,4166,558],{},[104,4168,766],{},[97,4170,4171,4173],{"accent":99},[60,4172,137],{},[104,4174,2849],{"stretchy":99},[104,4176,816],{"fence":99},[57,4178,4179,4181,4187,4189,4195],{},[104,4180,803],{"fence":99},[282,4182,4183,4185],{},[60,4184,102],{},[60,4186,558],{},[104,4188,766],{},[97,4190,4191,4193],{"accent":99},[60,4192,102],{},[104,4194,2849],{"stretchy":99},[104,4196,816],{"fence":99},[57,4198,4199,4213],{},[549,4200,4201,4203,4211],{},[104,4202,553],{},[57,4204,4205,4207,4209],{},[60,4206,558],{},[104,4208,131],{},[287,4210,289],{},[60,4212,917],{},[797,4214,4215,4235],{},[57,4216,4217,4219,4225,4227,4233],{},[104,4218,803],{"fence":99},[282,4220,4221,4223],{},[60,4222,137],{},[60,4224,558],{},[104,4226,766],{},[97,4228,4229,4231],{"accent":99},[60,4230,137],{},[104,4232,2849],{"stretchy":99},[104,4234,816],{"fence":99},[287,4236,303],{},[64,4238,4239],{"encoding":66},"k = \\frac{\n  \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right) \\left( y_{i} - \\overline{y} \\right)\n}{\n  \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right)^{2}\n}",[13,4241,4242,4243],{},"As a result, we get ",[45,4244,4246],{"className":4245},[48],[50,4247,4248],{"xmlns":52},[54,4249,4250,4276],{},[57,4251,4252,4254,4256,4270,4273],{},[60,4253,134],{},[104,4255,131],{},[899,4257,4258,4267],{},[57,4259,4260,4262,4264],{},[287,4261,289],{},[294,4263,296],{},[287,4265,4266],{},"517.6",[287,4268,4269],{},"1177.2",[104,4271,4272],{},"≈",[287,4274,4275],{},"1.29",[64,4277,4278],{"encoding":66},"k = \\frac{1\\ 517.6}{1177.2} \\approx 1.29",[13,4280,4281,4282,1349],{},"Now let's calculate ",[45,4283,4285],{"className":4284},[48],[50,4286,4287],{"xmlns":52},[54,4288,4289,4293],{},[57,4290,4291],{},[60,4292,143],{},[64,4294,143],{"encoding":66},[45,4296,4298],{"className":4297},[48],[50,4299,4300],{"xmlns":52,"display":122},[54,4301,4302,4345],{},[57,4303,4304,4306,4308,4314,4316,4318,4320,4326,4328,4330,4332,4334,4336,4338,4340,4342],{},[60,4305,143],{},[104,4307,131],{},[97,4309,4310,4312],{"accent":99},[60,4311,102],{},[104,4313,2849],{"stretchy":99},[104,4315,766],{},[60,4317,134],{},[104,4319,1311],{},[97,4321,4322,4324],{"accent":99},[60,4323,137],{},[104,4325,2849],{"stretchy":99},[104,4327,131],{},[287,4329,3941],{},[104,4331,766],{},[287,4333,4275],{},[104,4335,1311],{},[287,4337,3901],{},[104,4339,131],{},[104,4341,766],{},[287,4343,4344],{},"150.144",[64,4346,4347],{"encoding":66},"b = \\overline{y} - k \\cdot \\overline{x} = 73.8 - 1.29 \\cdot 173.6 = - 150.144",[13,4349,4350],{},"Now that we have all the coefficients, we can predict the weight for any\nheight. Let's say we have a basketball player who is 210 centimeters tall.\nLet's try to predict their weight:\\",[45,4352,4354],{"className":4353},[48],[50,4355,4356],{"xmlns":52,"display":122},[54,4357,4358,4376],{},[57,4359,4360,4363,4365,4367,4369,4371,4373],{},[287,4361,4362],{},"210",[104,4364,1311],{},[287,4366,4275],{},[104,4368,766],{},[287,4370,4344],{},[104,4372,4272],{},[287,4374,4375],{},"120.75",[64,4377,4378],{"encoding":66},"210 \\cdot 1.29 - 150.144  \\approx 120.75",[13,4380,4381],{},"Looks quite plausible.",[37,4383,4385],{"id":4384},"generalizing","Generalizing",[13,4387,4388],{},"Now that we know how to find coefficients for the one-dimensional case,\nlet's try to do it for the multidimensional case. Then our main formula\nwill be:",[45,4390,4392],{"className":4391},[48],[50,4393,4394],{"xmlns":52,"display":122},[54,4395,4396,4454],{},[57,4397,4398,4404,4406,4412,4418,4420,4426,4432,4434,4436,4438,4444,4450,4452],{},[97,4399,4400,4402],{"accent":99},[60,4401,102],{},[104,4403,106],{"stretchy":99},[104,4405,131],{},[282,4407,4408,4410],{},[60,4409,134],{},[287,4411,289],{},[282,4413,4414,4416],{},[60,4415,137],{},[287,4417,289],{},[104,4419,140],{},[282,4421,4422,4424],{},[60,4423,134],{},[287,4425,303],{},[282,4427,4428,4430],{},[60,4429,137],{},[287,4431,303],{},[104,4433,140],{},[104,4435,506],{},[104,4437,140],{},[282,4439,4440,4442],{},[60,4441,134],{},[60,4443,455],{},[282,4445,4446,4448],{},[60,4447,137],{},[60,4449,455],{},[104,4451,140],{},[60,4453,143],{},[64,4455,527],{"encoding":66},[13,4457,4458],{},"And the function we'll optimize:",[45,4460,4462],{"className":4461},[48],[50,4463,4464],{"xmlns":52,"display":122},[54,4465,4466,4568],{},[57,4467,4468,4482],{},[549,4469,4470,4472,4480],{},[104,4471,553],{},[57,4473,4474,4476,4478],{},[60,4475,558],{},[104,4477,131],{},[287,4479,289],{},[60,4481,917],{},[797,4483,4484,4566],{},[57,4485,4486,4488,4494,4496,4564],{},[104,4487,803],{"fence":99},[282,4489,4490,4492],{},[60,4491,102],{},[60,4493,558],{},[104,4495,766],{},[57,4497,4498,4500,4506,4516,4518,4524,4534,4536,4538,4540,4546,4548,4558,4560,4562],{},[104,4499,803],{"fence":99},[282,4501,4502,4504],{},[60,4503,134],{},[287,4505,289],{},[282,4507,4508,4510],{},[60,4509,137],{},[57,4511,4512,4514],{},[287,4513,289],{},[60,4515,558],{},[104,4517,140],{},[282,4519,4520,4522],{},[60,4521,134],{},[287,4523,303],{},[282,4525,4526,4528],{},[60,4527,137],{},[57,4529,4530,4532],{},[287,4531,303],{},[60,4533,558],{},[104,4535,140],{},[104,4537,506],{},[104,4539,140],{},[282,4541,4542,4544],{},[60,4543,134],{},[60,4545,455],{},[104,4547,869],{},[282,4549,4550,4552],{},[60,4551,137],{},[57,4553,4554,4556],{},[60,4555,455],{},[60,4557,558],{},[104,4559,140],{},[60,4561,143],{},[104,4563,816],{"fence":99},[104,4565,816],{"fence":99},[287,4567,303],{},[64,4569,4570],{"encoding":66},"\\sum_{i = 1}^{n} \\left( y_{i} - \\left( k_{1}x_{1i} + k_{2}x_{2i} + \\ldots + k_{m}*x_{mi} + b \\right) \\right)^{2}",[13,4572,4573,4574,4587,4588,4605,4606,4619,4620,4638,4639,4656],{},"The optimization itself follows the same principle. We compute ",[45,4575,4577],{"className":4576},[48],[50,4578,4579],{"xmlns":52},[54,4580,4581,4585],{},[57,4582,4583],{},[60,4584,455],{},[64,4586,455],{"encoding":66},"\nderivatives with respect to each coefficient ",[45,4589,4591],{"className":4590},[48],[50,4592,4593],{"xmlns":52},[54,4594,4595,4603],{},[57,4596,4597],{},[282,4598,4599,4601],{},[60,4600,134],{},[60,4602,558],{},[64,4604,604],{"encoding":66},", and one derivative\nwith respect to ",[45,4607,4609],{"className":4608},[48],[50,4610,4611],{"xmlns":52},[54,4612,4613,4617],{},[57,4614,4615],{},[60,4616,143],{},[64,4618,143],{"encoding":66},". As a result, we get a system of ",[45,4621,4623],{"className":4622},[48],[50,4624,4625],{"xmlns":52},[54,4626,4627,4635],{},[57,4628,4629,4631,4633],{},[60,4630,455],{},[104,4632,140],{},[287,4634,289],{},[64,4636,4637],{"encoding":66},"m + 1"," equations\nwith ",[45,4640,4642],{"className":4641},[48],[50,4643,4644],{"xmlns":52},[54,4645,4646,4654],{},[57,4647,4648,4650,4652],{},[60,4649,455],{},[104,4651,140],{},[287,4653,289],{},[64,4655,4637],{"encoding":66}," unknowns.",[13,4658,4659],{},"Solving such a system is already beyond the scope of school mathematics,\nbut it's entirely feasible. As a result, we'll find all our coefficients\nin the same way.",[13,4661,4662],{},"If in the case of one argument our function defined a straight line, in\nthe case of two arguments it will be a plane, and with more arguments — a\nhyperplane.",[37,4664,4666],{"id":4665},"not-everything-is-so-rosy","Not Everything Is So Rosy",[13,4668,4669],{},"It would seem that if training is so fast and simple (just calculate a\nfew formulas), why isn't linear regression used everywhere? The answer\nlies in the word \"linear.\" And now let's try to figure out why.",[13,4671,4672,4673,4686,4687,4700],{},"Let's consider the simplest case again. The dependence of ",[45,4674,4676],{"className":4675},[48],[50,4677,4678],{"xmlns":52},[54,4679,4680,4684],{},[57,4681,4682],{},[60,4683,102],{},[64,4685,102],{"encoding":66}," on ",[45,4688,4690],{"className":4689},[48],[50,4691,4692],{"xmlns":52},[54,4693,4694,4698],{},[57,4695,4696],{},[60,4697,137],{},[64,4699,137],{"encoding":66}," is\nlinear. This means we can make good predictions only if the real-world\ndependency between the data is linear or close to it. If there are\nsignificant deviations from a linear relationship, the linear regression\nmodel will still produce some result, but it will differ substantially\nfrom reality.",[13,4702,4703],{},[1051,4704],{"alt":4705,"src":4706},"The problem with nonlinear dependency","/img/blog/ml-basic-2/image3.png",[13,4708,4709],{},"In the image above, we see a nonlinear distribution of real data (blue\ndots). The model was trained, trying to minimize the distances between\nthe real data and the predictions (red dots). But even though the\ndistance is the minimum possible for this case, the model's prediction\ncan be either relatively accurate (2nd and 4th points) or very inaccurate\ndepending on the region.",[13,4711,4712],{},"Variants with two or more arguments are also subject to this limitation\n(but instead of a straight line, it's a plane or hyperplane).",[13,4714,4715],{},"It is precisely because of this characteristic that linear regression is\nused relatively rarely. And it's only applied after verifying that the\ndata indeed has a linear relationship. However, when such a relationship\nexists, we get a very fast tool — much faster than any other machine\nlearning model.",[13,4717,4718],{},"For example, linear regression is very often used in economics, where the\nlinearity of the relationship is well known: forecasting sales, assets,\nGDP, and so on. It is also used by banks and insurance companies. In this\nfield, it's important not only to make a prediction but also to explain it.\nAnd linear regression is the best fit for this. All of its parameters are\nvisible and understandable. Unlike, for example, neural networks, which,\nas we'll see in the following articles of this series, are a \"black box\"\nto an outside observer.",[37,4720,4722],{"id":4721},"afterword","Afterword",[13,4724,4725],{},"As promised, I'm showing how the coefficient formulas can be simplified.\nFor this, we'll need a couple of mathematical tricks and some knowledge\nof statistics.",[13,4727,4728],{},"Let me remind you of the original expression:",[45,4730,4732],{"className":4731},[48],[50,4733,4734],{"xmlns":52,"display":122},[54,4735,4736,4842],{},[57,4737,4738,4740,4754,4788,4790,4804,4838,4840],{},[60,4739,134],{},[549,4741,4742,4744,4752],{},[104,4743,553],{},[57,4745,4746,4748,4750],{},[60,4747,558],{},[104,4749,131],{},[287,4751,289],{},[60,4753,917],{},[57,4755,4756,4758,4778,4780,4786],{},[104,4757,803],{"fence":99},[57,4759,4760,4762,4768,4770,4776],{},[104,4761,803],{"fence":99},[282,4763,4764,4766],{},[60,4765,137],{},[60,4767,558],{},[104,4769,766],{},[97,4771,4772,4774],{"accent":99},[60,4773,137],{},[104,4775,2849],{"stretchy":99},[104,4777,816],{"fence":99},[104,4779,1311],{},[282,4781,4782,4784],{},[60,4783,137],{},[60,4785,558],{},[104,4787,816],{"fence":99},[104,4789,140],{},[549,4791,4792,4794,4802],{},[104,4793,553],{},[57,4795,4796,4798,4800],{},[60,4797,558],{},[104,4799,131],{},[287,4801,289],{},[60,4803,917],{},[57,4805,4806,4808,4828,4830,4836],{},[104,4807,803],{"fence":99},[57,4809,4810,4812,4818,4820,4826],{},[104,4811,803],{"fence":99},[97,4813,4814,4816],{"accent":99},[60,4815,102],{},[104,4817,2849],{"stretchy":99},[104,4819,766],{},[282,4821,4822,4824],{},[60,4823,102],{},[60,4825,558],{},[104,4827,816],{"fence":99},[104,4829,1311],{},[282,4831,4832,4834],{},[60,4833,137],{},[60,4835,558],{},[104,4837,816],{"fence":99},[104,4839,131],{},[287,4841,780],{},[64,4843,3277],{"encoding":66},[13,4845,4846,4847,4864],{},"We'll consider only its first term. Let's add the mean value to ",[45,4848,4850],{"className":4849},[48],[50,4851,4852],{"xmlns":52},[54,4853,4854,4862],{},[57,4855,4856],{},[282,4857,4858,4860],{},[60,4859,137],{},[60,4861,558],{},[64,4863,1171],{"encoding":66},",\nand immediately subtract it for compensation:",[45,4866,4868],{"className":4867},[48],[50,4869,4870],{"xmlns":52,"display":122},[54,4871,4872,4998],{},[57,4873,4874,4876,4890,4924,4926,4928,4942],{},[60,4875,134],{},[549,4877,4878,4880,4888],{},[104,4879,553],{},[57,4881,4882,4884,4886],{},[60,4883,558],{},[104,4885,131],{},[287,4887,289],{},[60,4889,917],{},[57,4891,4892,4894,4914,4916,4922],{},[104,4893,803],{"fence":99},[57,4895,4896,4898,4904,4906,4912],{},[104,4897,803],{"fence":99},[282,4899,4900,4902],{},[60,4901,137],{},[60,4903,558],{},[104,4905,766],{},[97,4907,4908,4910],{"accent":99},[60,4909,137],{},[104,4911,2849],{"stretchy":99},[104,4913,816],{"fence":99},[104,4915,1311],{},[282,4917,4918,4920],{},[60,4919,137],{},[60,4921,558],{},[104,4923,816],{"fence":99},[104,4925,131],{},[60,4927,134],{},[549,4929,4930,4932,4940],{},[104,4931,553],{},[57,4933,4934,4936,4938],{},[60,4935,558],{},[104,4937,131],{},[287,4939,289],{},[60,4941,917],{},[57,4943,4944,4946,4966,4968,4996],{},[104,4945,803],{"fence":99},[57,4947,4948,4950,4956,4958,4964],{},[104,4949,803],{"fence":99},[282,4951,4952,4954],{},[60,4953,137],{},[60,4955,558],{},[104,4957,766],{},[97,4959,4960,4962],{"accent":99},[60,4961,137],{},[104,4963,2849],{"stretchy":99},[104,4965,816],{"fence":99},[104,4967,1311],{},[57,4969,4970,4972,4978,4980,4986,4988,4994],{},[104,4971,803],{"fence":99},[282,4973,4974,4976],{},[60,4975,137],{},[60,4977,558],{},[104,4979,140],{},[97,4981,4982,4984],{"accent":99},[60,4983,137],{},[104,4985,2849],{"stretchy":99},[104,4987,766],{},[97,4989,4990,4992],{"accent":99},[60,4991,137],{},[104,4993,2849],{"stretchy":99},[104,4995,816],{"fence":99},[104,4997,816],{"fence":99},[64,4999,5000],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( \\left( x_{i} - \\overline{x} \\right) \\cdot x_{i} \\right) =\nk \\sum_{i = 1}^{n} \\left( \\left(x_{i} - \\overline{x} \\right) \\cdot \\left(x_{i} + \\overline{x} - \\overline{x} \\right) \\right)",[13,5002,5003],{},"Expand the brackets:",[45,5005,5007],{"className":5006},[48],[50,5008,5009],{"xmlns":52,"display":122},[54,5010,5011,5183],{},[57,5012,5013,5015,5029,5085,5087,5089,5103],{},[60,5014,134],{},[549,5016,5017,5019,5027],{},[104,5018,553],{},[57,5020,5021,5023,5025],{},[60,5022,558],{},[104,5024,131],{},[287,5026,289],{},[60,5028,917],{},[57,5030,5031,5033,5053,5055,5083],{},[104,5032,803],{"fence":99},[57,5034,5035,5037,5043,5045,5051],{},[104,5036,803],{"fence":99},[282,5038,5039,5041],{},[60,5040,137],{},[60,5042,558],{},[104,5044,766],{},[97,5046,5047,5049],{"accent":99},[60,5048,137],{},[104,5050,2849],{"stretchy":99},[104,5052,816],{"fence":99},[104,5054,1311],{},[57,5056,5057,5059,5065,5067,5073,5075,5081],{},[104,5058,803],{"fence":99},[282,5060,5061,5063],{},[60,5062,137],{},[60,5064,558],{},[104,5066,140],{},[97,5068,5069,5071],{"accent":99},[60,5070,137],{},[104,5072,2849],{"stretchy":99},[104,5074,766],{},[97,5076,5077,5079],{"accent":99},[60,5078,137],{},[104,5080,2849],{"stretchy":99},[104,5082,816],{"fence":99},[104,5084,816],{"fence":99},[104,5086,131],{},[60,5088,134],{},[549,5090,5091,5093,5101],{},[104,5092,553],{},[57,5094,5095,5097,5099],{},[60,5096,558],{},[104,5098,131],{},[287,5100,289],{},[60,5102,917],{},[57,5104,5105,5107,5115,5117,5123,5129,5131,5137,5143,5145,5151,5157,5159,5169,5171,5181],{},[104,5106,803],{"fence":99},[1737,5108,5109,5111,5113],{},[60,5110,137],{},[60,5112,558],{},[287,5114,303],{},[104,5116,140],{},[282,5118,5119,5121],{},[60,5120,137],{},[60,5122,558],{},[97,5124,5125,5127],{"accent":99},[60,5126,137],{},[104,5128,2849],{"stretchy":99},[104,5130,766],{},[282,5132,5133,5135],{},[60,5134,137],{},[60,5136,558],{},[97,5138,5139,5141],{"accent":99},[60,5140,137],{},[104,5142,2849],{"stretchy":99},[104,5144,766],{},[282,5146,5147,5149],{},[60,5148,137],{},[60,5150,558],{},[97,5152,5153,5155],{"accent":99},[60,5154,137],{},[104,5156,2849],{"stretchy":99},[104,5158,766],{},[797,5160,5161,5167],{},[97,5162,5163,5165],{"accent":99},[60,5164,137],{},[104,5166,2849],{"stretchy":99},[287,5168,303],{},[104,5170,140],{},[797,5172,5173,5179],{},[97,5174,5175,5177],{"accent":99},[60,5176,137],{},[104,5178,2849],{"stretchy":99},[287,5180,303],{},[104,5182,816],{"fence":99},[64,5184,5185],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( \\left( x_{i} - \\overline{x} \\right) \\cdot \\left(x_{i} + \\overline{x} - \\overline{x} \\right) \\right) =\nk \\sum_{i = 1}^{n} \\left( x_{i}^{2} + x_{i} \\overline{x} - x_{i} \\overline{x} - x_{i} \\overline{x} - {\\overline{x}}^{2} + {\\overline{x}}^{2} \\right)",[13,5187,5188],{},"Divide the sum into two parts:",[45,5190,5192],{"className":5191},[48],[50,5193,5194],{"xmlns":52,"display":122},[54,5195,5196,5414],{},[57,5197,5198,5200,5214,5294,5296,5298,5312,5366,5368,5370,5384],{},[60,5199,134],{},[549,5201,5202,5204,5212],{},[104,5203,553],{},[57,5205,5206,5208,5210],{},[60,5207,558],{},[104,5209,131],{},[287,5211,289],{},[60,5213,917],{},[57,5215,5216,5218,5226,5228,5234,5240,5242,5248,5254,5256,5262,5268,5270,5280,5282,5292],{},[104,5217,803],{"fence":99},[1737,5219,5220,5222,5224],{},[60,5221,137],{},[60,5223,558],{},[287,5225,303],{},[104,5227,140],{},[282,5229,5230,5232],{},[60,5231,137],{},[60,5233,558],{},[97,5235,5236,5238],{"accent":99},[60,5237,137],{},[104,5239,2849],{"stretchy":99},[104,5241,766],{},[282,5243,5244,5246],{},[60,5245,137],{},[60,5247,558],{},[97,5249,5250,5252],{"accent":99},[60,5251,137],{},[104,5253,2849],{"stretchy":99},[104,5255,766],{},[282,5257,5258,5260],{},[60,5259,137],{},[60,5261,558],{},[97,5263,5264,5266],{"accent":99},[60,5265,137],{},[104,5267,2849],{"stretchy":99},[104,5269,766],{},[797,5271,5272,5278],{},[97,5273,5274,5276],{"accent":99},[60,5275,137],{},[104,5277,2849],{"stretchy":99},[287,5279,303],{},[104,5281,140],{},[797,5283,5284,5290],{},[97,5285,5286,5288],{"accent":99},[60,5287,137],{},[104,5289,2849],{"stretchy":99},[287,5291,303],{},[104,5293,816],{"fence":99},[104,5295,131],{},[60,5297,134],{},[549,5299,5300,5302,5310],{},[104,5301,553],{},[57,5303,5304,5306,5308],{},[60,5305,558],{},[104,5307,131],{},[287,5309,289],{},[60,5311,917],{},[57,5313,5314,5316,5324,5326,5332,5338,5340,5346,5352,5354,5364],{},[104,5315,803],{"fence":99},[1737,5317,5318,5320,5322],{},[60,5319,137],{},[60,5321,558],{},[287,5323,303],{},[104,5325,766],{},[282,5327,5328,5330],{},[60,5329,137],{},[60,5331,558],{},[97,5333,5334,5336],{"accent":99},[60,5335,137],{},[104,5337,2849],{"stretchy":99},[104,5339,766],{},[282,5341,5342,5344],{},[60,5343,137],{},[60,5345,558],{},[97,5347,5348,5350],{"accent":99},[60,5349,137],{},[104,5351,2849],{"stretchy":99},[104,5353,140],{},[797,5355,5356,5362],{},[97,5357,5358,5360],{"accent":99},[60,5359,137],{},[104,5361,2849],{"stretchy":99},[287,5363,303],{},[104,5365,816],{"fence":99},[104,5367,140],{},[60,5369,134],{},[549,5371,5372,5374,5382],{},[104,5373,553],{},[57,5375,5376,5378,5380],{},[60,5377,558],{},[104,5379,131],{},[287,5381,289],{},[60,5383,917],{},[57,5385,5386,5388,5394,5400,5402,5412],{},[104,5387,803],{"fence":99},[282,5389,5390,5392],{},[60,5391,137],{},[60,5393,558],{},[97,5395,5396,5398],{"accent":99},[60,5397,137],{},[104,5399,2849],{"stretchy":99},[104,5401,766],{},[797,5403,5404,5410],{},[97,5405,5406,5408],{"accent":99},[60,5407,137],{},[104,5409,2849],{"stretchy":99},[287,5411,303],{},[104,5413,816],{"fence":99},[64,5415,5416],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( x_{i}^{2} + x_{i} \\overline{x} - x_{i} \\overline{x} - x_{i} \\overline{x} - {\\overline{x}}^{2} + {\\overline{x}}^{2} \\right) =\nk \\sum_{i = 1}^{n} \\left( x_{i}^{2} - x_{i} \\overline{x} - x_{i} \\overline{x} + {\\overline{x}}^{2} \\right) +\nk \\sum_{i = 1}^{n} \\left( x_{i}\\overline{x} - {\\overline{x}}^{2} \\right)",[13,5418,5419],{},"In the first part, we see the formula for the square of a difference:",[45,5421,5423],{"className":5422},[48],[50,5424,5425],{"xmlns":52,"display":122},[54,5426,5427,5601],{},[57,5428,5429,5431,5445,5499,5501,5503,5517,5559,5561,5563,5577],{},[60,5430,134],{},[549,5432,5433,5435,5443],{},[104,5434,553],{},[57,5436,5437,5439,5441],{},[60,5438,558],{},[104,5440,131],{},[287,5442,289],{},[60,5444,917],{},[57,5446,5447,5449,5457,5459,5465,5471,5473,5479,5485,5487,5497],{},[104,5448,803],{"fence":99},[1737,5450,5451,5453,5455],{},[60,5452,137],{},[60,5454,558],{},[287,5456,303],{},[104,5458,766],{},[282,5460,5461,5463],{},[60,5462,137],{},[60,5464,558],{},[97,5466,5467,5469],{"accent":99},[60,5468,137],{},[104,5470,2849],{"stretchy":99},[104,5472,766],{},[282,5474,5475,5477],{},[60,5476,137],{},[60,5478,558],{},[97,5480,5481,5483],{"accent":99},[60,5482,137],{},[104,5484,2849],{"stretchy":99},[104,5486,140],{},[797,5488,5489,5495],{},[97,5490,5491,5493],{"accent":99},[60,5492,137],{},[104,5494,2849],{"stretchy":99},[287,5496,303],{},[104,5498,816],{"fence":99},[104,5500,131],{},[60,5502,134],{},[549,5504,5505,5507,5515],{},[104,5506,553],{},[57,5508,5509,5511,5513],{},[60,5510,558],{},[104,5512,131],{},[287,5514,289],{},[60,5516,917],{},[57,5518,5519,5521,5529,5531,5533,5539,5545,5547,5557],{},[104,5520,803],{"fence":99},[1737,5522,5523,5525,5527],{},[60,5524,137],{},[60,5526,558],{},[287,5528,303],{},[104,5530,766],{},[287,5532,303],{},[282,5534,5535,5537],{},[60,5536,137],{},[60,5538,558],{},[97,5540,5541,5543],{"accent":99},[60,5542,137],{},[104,5544,2849],{"stretchy":99},[104,5546,140],{},[797,5548,5549,5555],{},[97,5550,5551,5553],{"accent":99},[60,5552,137],{},[104,5554,2849],{"stretchy":99},[287,5556,303],{},[104,5558,816],{"fence":99},[104,5560,131],{},[60,5562,134],{},[549,5564,5565,5567,5575],{},[104,5566,553],{},[57,5568,5569,5571,5573],{},[60,5570,558],{},[104,5572,131],{},[287,5574,289],{},[60,5576,917],{},[797,5578,5579,5599],{},[57,5580,5581,5583,5589,5591,5597],{},[104,5582,803],{"fence":99},[282,5584,5585,5587],{},[60,5586,137],{},[60,5588,558],{},[104,5590,766],{},[97,5592,5593,5595],{"accent":99},[60,5594,137],{},[104,5596,2849],{"stretchy":99},[104,5598,816],{"fence":99},[287,5600,303],{},[64,5602,5603],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( x_{i}^{2} - x_{i} \\overline{x} - x_{i}\\overline{x} + {\\overline{x}}^{2} \\right) =\nk \\sum_{i = 1}^{n} \\left( x_{i}^{2} - 2 x_{i} \\overline{x} + {\\overline{x}}^{2} \\right) = k \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right)^{2}",[13,5605,5606,5607,5625],{},"Now let's deal with the second part. We factor out ",[45,5608,5610],{"className":5609},[48],[50,5611,5612],{"xmlns":52},[54,5613,5614,5622],{},[57,5615,5616],{},[97,5617,5618,5620],{"accent":99},[60,5619,137],{},[104,5621,2849],{"stretchy":99},[64,5623,5624],{"encoding":66},"\\overline{x}"," from\nthe sum:",[45,5627,5629],{"className":5628},[48],[50,5630,5631],{"xmlns":52,"display":122},[54,5632,5633,5725],{},[57,5634,5635,5637,5651,5681,5683,5685,5691,5705],{},[60,5636,134],{},[549,5638,5639,5641,5649],{},[104,5640,553],{},[57,5642,5643,5645,5647],{},[60,5644,558],{},[104,5646,131],{},[287,5648,289],{},[60,5650,917],{},[57,5652,5653,5655,5661,5667,5669,5679],{},[104,5654,803],{"fence":99},[282,5656,5657,5659],{},[60,5658,137],{},[60,5660,558],{},[97,5662,5663,5665],{"accent":99},[60,5664,137],{},[104,5666,2849],{"stretchy":99},[104,5668,766],{},[797,5670,5671,5677],{},[97,5672,5673,5675],{"accent":99},[60,5674,137],{},[104,5676,2849],{"stretchy":99},[287,5678,303],{},[104,5680,816],{"fence":99},[104,5682,131],{},[60,5684,134],{},[97,5686,5687,5689],{"accent":99},[60,5688,137],{},[104,5690,2849],{"stretchy":99},[549,5692,5693,5695,5703],{},[104,5694,553],{},[57,5696,5697,5699,5701],{},[60,5698,558],{},[104,5700,131],{},[287,5702,289],{},[60,5704,917],{},[57,5706,5707,5709,5715,5717,5723],{},[104,5708,803],{"fence":99},[282,5710,5711,5713],{},[60,5712,137],{},[60,5714,558],{},[104,5716,766],{},[97,5718,5719,5721],{"accent":99},[60,5720,137],{},[104,5722,2849],{"stretchy":99},[104,5724,816],{"fence":99},[64,5726,5727],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( x_{i}\\overline{x} - {\\overline{x}}^{2} \\right) =\nk \\overline{x}\\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right)",[13,5729,5730,5731,5761],{},"And now comes that very trick from statistics.\n",[45,5732,5734],{"className":5733},[48],[50,5735,5736],{"xmlns":52},[54,5737,5738,5758],{},[57,5739,5740,5742,5748,5750,5756],{},[104,5741,803],{"fence":99},[282,5743,5744,5746],{},[60,5745,137],{},[60,5747,558],{},[104,5749,766],{},[97,5751,5752,5754],{"accent":99},[60,5753,137],{},[104,5755,2849],{"stretchy":99},[104,5757,816],{"fence":99},[64,5759,5760],{"encoding":66},"\\left( x_{i} - \\overline{x} \\right)"," is the deviation from the mean.\nAnd the most interesting thing is that the sum of such deviations is\nalways zero (this follows from statistics):",[45,5763,5765],{"className":5764},[48],[50,5766,5767],{"xmlns":52,"display":122},[54,5768,5769,5809],{},[57,5770,5771,5785,5805,5807],{},[549,5772,5773,5775,5783],{},[104,5774,553],{},[57,5776,5777,5779,5781],{},[60,5778,558],{},[104,5780,131],{},[287,5782,289],{},[60,5784,917],{},[57,5786,5787,5789,5795,5797,5803],{},[104,5788,803],{"fence":99},[282,5790,5791,5793],{},[60,5792,137],{},[60,5794,558],{},[104,5796,766],{},[97,5798,5799,5801],{"accent":99},[60,5800,137],{},[104,5802,2849],{"stretchy":99},[104,5804,816],{"fence":99},[104,5806,131],{},[287,5808,780],{},[64,5810,5811],{"encoding":66},"\\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right) = 0",[13,5813,5814],{},"So in this part of the equation, only one term remains.",[13,5816,5817],{},"Let's return to the original equation and process its second part in the\nsame way:",[45,5819,5821],{"className":5820},[48],[50,5822,5823],{"xmlns":52,"display":122},[54,5824,5825,6047],{},[57,5826,5827,5841,5875,5877,5891,5947,5949,5963,5983,5985,6005,6007,6013,6027],{},[549,5828,5829,5831,5839],{},[104,5830,553],{},[57,5832,5833,5835,5837],{},[60,5834,558],{},[104,5836,131],{},[287,5838,289],{},[60,5840,917],{},[57,5842,5843,5845,5865,5867,5873],{},[104,5844,803],{"fence":99},[57,5846,5847,5849,5855,5857,5863],{},[104,5848,803],{"fence":99},[97,5850,5851,5853],{"accent":99},[60,5852,102],{},[104,5854,2849],{"stretchy":99},[104,5856,766],{},[282,5858,5859,5861],{},[60,5860,102],{},[60,5862,558],{},[104,5864,816],{"fence":99},[104,5866,1311],{},[282,5868,5869,5871],{},[60,5870,137],{},[60,5872,558],{},[104,5874,816],{"fence":99},[104,5876,131],{},[549,5878,5879,5881,5889],{},[104,5880,553],{},[57,5882,5883,5885,5887],{},[60,5884,558],{},[104,5886,131],{},[287,5888,289],{},[60,5890,917],{},[57,5892,5893,5895,5915,5917,5945],{},[104,5894,803],{"fence":99},[57,5896,5897,5899,5905,5907,5913],{},[104,5898,803],{"fence":99},[97,5900,5901,5903],{"accent":99},[60,5902,102],{},[104,5904,2849],{"stretchy":99},[104,5906,766],{},[282,5908,5909,5911],{},[60,5910,102],{},[60,5912,558],{},[104,5914,816],{"fence":99},[104,5916,1311],{},[57,5918,5919,5921,5927,5929,5935,5937,5943],{},[104,5920,803],{"fence":99},[282,5922,5923,5925],{},[60,5924,137],{},[60,5926,558],{},[104,5928,766],{},[97,5930,5931,5933],{"accent":99},[60,5932,137],{},[104,5934,2849],{"stretchy":99},[104,5936,140],{},[97,5938,5939,5941],{"accent":99},[60,5940,137],{},[104,5942,2849],{"stretchy":99},[104,5944,816],{"fence":99},[104,5946,816],{"fence":99},[104,5948,131],{},[549,5950,5951,5953,5961],{},[104,5952,553],{},[57,5954,5955,5957,5959],{},[60,5956,558],{},[104,5958,131],{},[287,5960,289],{},[60,5962,917],{},[57,5964,5965,5967,5973,5975,5981],{},[104,5966,803],{"fence":99},[97,5968,5969,5971],{"accent":99},[60,5970,102],{},[104,5972,2849],{"stretchy":99},[104,5974,766],{},[282,5976,5977,5979],{},[60,5978,102],{},[60,5980,558],{},[104,5982,816],{"fence":99},[104,5984,1311],{},[57,5986,5987,5989,5995,5997,6003],{},[104,5988,803],{"fence":99},[282,5990,5991,5993],{},[60,5992,137],{},[60,5994,558],{},[104,5996,766],{},[97,5998,5999,6001],{"accent":99},[60,6000,137],{},[104,6002,2849],{"stretchy":99},[104,6004,816],{"fence":99},[104,6006,140],{},[97,6008,6009,6011],{"accent":99},[60,6010,137],{},[104,6012,2849],{"stretchy":99},[549,6014,6015,6017,6025],{},[104,6016,553],{},[57,6018,6019,6021,6023],{},[60,6020,558],{},[104,6022,131],{},[287,6024,289],{},[60,6026,917],{},[57,6028,6029,6031,6037,6039,6045],{},[104,6030,803],{"fence":99},[97,6032,6033,6035],{"accent":99},[60,6034,102],{},[104,6036,2849],{"stretchy":99},[104,6038,766],{},[282,6040,6041,6043],{},[60,6042,102],{},[60,6044,558],{},[104,6046,816],{"fence":99},[64,6048,6049],{"encoding":66},"\\sum_{i = 1}^{n} \\left( \\left( \\overline{y} - y_{i} \\right) \\cdot x_{i} \\right) =\n\\sum_{i = 1}^{n} \\left( \\left( \\overline{y} - y_{i} \\right) \\cdot \\left( x_{i} - \\overline{x} + \\overline{x} \\right) \\right) =\n\\sum_{i = 1}^{n} \\left( \\overline{y} - y_{i} \\right) \\cdot \\left( x_{i} - \\overline{x} \\right) +\n\\overline{x} \\sum_{i = 1}^{n} \\left( \\overline{y} - y_{i} \\right)",[13,6051,6052],{},"The second term will also become zero, and we get the final equation:",[45,6054,6056],{"className":6055},[48],[50,6057,6058],{"xmlns":52,"display":122},[54,6059,6060,6164],{},[57,6061,6062,6064,6078,6102,6104,6118,6138,6140,6160,6162],{},[60,6063,134],{},[549,6065,6066,6068,6076],{},[104,6067,553],{},[57,6069,6070,6072,6074],{},[60,6071,558],{},[104,6073,131],{},[287,6075,289],{},[60,6077,917],{},[797,6079,6080,6100],{},[57,6081,6082,6084,6090,6092,6098],{},[104,6083,803],{"fence":99},[282,6085,6086,6088],{},[60,6087,137],{},[60,6089,558],{},[104,6091,766],{},[97,6093,6094,6096],{"accent":99},[60,6095,137],{},[104,6097,2849],{"stretchy":99},[104,6099,816],{"fence":99},[287,6101,303],{},[104,6103,140],{},[549,6105,6106,6108,6116],{},[104,6107,553],{},[57,6109,6110,6112,6114],{},[60,6111,558],{},[104,6113,131],{},[287,6115,289],{},[60,6117,917],{},[57,6119,6120,6122,6128,6130,6136],{},[104,6121,803],{"fence":99},[97,6123,6124,6126],{"accent":99},[60,6125,102],{},[104,6127,2849],{"stretchy":99},[104,6129,766],{},[282,6131,6132,6134],{},[60,6133,102],{},[60,6135,558],{},[104,6137,816],{"fence":99},[104,6139,1311],{},[57,6141,6142,6144,6150,6152,6158],{},[104,6143,803],{"fence":99},[282,6145,6146,6148],{},[60,6147,137],{},[60,6149,558],{},[104,6151,766],{},[97,6153,6154,6156],{"accent":99},[60,6155,137],{},[104,6157,2849],{"stretchy":99},[104,6159,816],{"fence":99},[104,6161,131],{},[287,6163,780],{},[64,6165,6166],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right)^{2} +\n\\sum_{i = 1}^{n} \\left( \\overline{y} - y_{i} \\right)\\cdot \\left( x_{i} - \\overline{x} \\right) = 0",[13,6168,6169],{},"From here, moving the second term to the right, changing its sign, and\ndividing, we get:",[45,6171,6173],{"className":6172},[48],[50,6174,6175],{"xmlns":52,"display":122},[54,6176,6177,6277],{},[57,6178,6179,6181,6195,6219,6221,6235,6255,6257],{},[60,6180,134],{},[549,6182,6183,6185,6193],{},[104,6184,553],{},[57,6186,6187,6189,6191],{},[60,6188,558],{},[104,6190,131],{},[287,6192,289],{},[60,6194,917],{},[797,6196,6197,6217],{},[57,6198,6199,6201,6207,6209,6215],{},[104,6200,803],{"fence":99},[282,6202,6203,6205],{},[60,6204,137],{},[60,6206,558],{},[104,6208,766],{},[97,6210,6211,6213],{"accent":99},[60,6212,137],{},[104,6214,2849],{"stretchy":99},[104,6216,816],{"fence":99},[287,6218,303],{},[104,6220,131],{},[549,6222,6223,6225,6233],{},[104,6224,553],{},[57,6226,6227,6229,6231],{},[60,6228,558],{},[104,6230,131],{},[287,6232,289],{},[60,6234,917],{},[57,6236,6237,6239,6245,6247,6253],{},[104,6238,803],{"fence":99},[282,6240,6241,6243],{},[60,6242,102],{},[60,6244,558],{},[104,6246,766],{},[97,6248,6249,6251],{"accent":99},[60,6250,102],{},[104,6252,2849],{"stretchy":99},[104,6254,816],{"fence":99},[104,6256,1311],{},[57,6258,6259,6261,6267,6269,6275],{},[104,6260,803],{"fence":99},[282,6262,6263,6265],{},[60,6264,137],{},[60,6266,558],{},[104,6268,766],{},[97,6270,6271,6273],{"accent":99},[60,6272,137],{},[104,6274,2849],{"stretchy":99},[104,6276,816],{"fence":99},[64,6278,6279],{"encoding":66},"k \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right)^{2} =\n\\sum_{i = 1}^{n} \\left( y_{i} - \\overline{y} \\right) \\cdot \\left( x_{i} - \\overline{x} \\right)",[45,6281,6283],{"className":6282},[48],[50,6284,6285],{"xmlns":52,"display":122},[54,6286,6287,6393],{},[57,6288,6289,6291,6293],{},[60,6290,134],{},[104,6292,131],{},[899,6294,6295,6353],{},[57,6296,6297,6311,6331,6333],{},[549,6298,6299,6301,6309],{},[104,6300,553],{},[57,6302,6303,6305,6307],{},[60,6304,558],{},[104,6306,131],{},[287,6308,289],{},[60,6310,917],{},[57,6312,6313,6315,6321,6323,6329],{},[104,6314,803],{"fence":99},[282,6316,6317,6319],{},[60,6318,102],{},[60,6320,558],{},[104,6322,766],{},[97,6324,6325,6327],{"accent":99},[60,6326,102],{},[104,6328,2849],{"stretchy":99},[104,6330,816],{"fence":99},[104,6332,869],{},[57,6334,6335,6337,6343,6345,6351],{},[104,6336,803],{"fence":99},[282,6338,6339,6341],{},[60,6340,137],{},[60,6342,558],{},[104,6344,766],{},[97,6346,6347,6349],{"accent":99},[60,6348,137],{},[104,6350,2849],{"stretchy":99},[104,6352,816],{"fence":99},[57,6354,6355,6369],{},[549,6356,6357,6359,6367],{},[104,6358,553],{},[57,6360,6361,6363,6365],{},[60,6362,558],{},[104,6364,131],{},[287,6366,289],{},[60,6368,917],{},[797,6370,6371,6391],{},[57,6372,6373,6375,6381,6383,6389],{},[104,6374,803],{"fence":99},[282,6376,6377,6379],{},[60,6378,137],{},[60,6380,558],{},[104,6382,766],{},[97,6384,6385,6387],{"accent":99},[60,6386,137],{},[104,6388,2849],{"stretchy":99},[104,6390,816],{"fence":99},[287,6392,303],{},[64,6394,6395],{"encoding":66},"k = \\frac{\n  \\sum_{i = 1}^{n} \\left( y_{i} - \\overline{y} \\right)*\\left( x_{i} - \\overline{x} \\right)\n}{\n  \\sum_{i = 1}^{n} \\left( x_{i} - \\overline{x} \\right)^{2}\n}",[6397,6398,6400,6440,6453,6466,6479,6585,6598,6680],"faq",{"title":6399},"Questions and Answers",[6401,6402,6404,6411],"faq-item",{"value":6403},"item-1",[6405,6406,6408],"template",{"v-slot:question":6407},"",[13,6409,6410],{},"What does \"linear\" mean in linear regression?",[6405,6412,6413],{"v-slot:answer":6407},[13,6414,6415,6416,6439],{},"It means the model assumes a linear (straight-line) relationship between\nthe input data and the output. The prediction formula is the equation of\na straight line ",[45,6417,6419],{"className":6418},[48],[50,6420,6421],{"xmlns":52},[54,6422,6423,6437],{},[57,6424,6425,6427,6429,6431,6433,6435],{},[60,6426,102],{},[104,6428,131],{},[60,6430,134],{},[60,6432,137],{},[104,6434,140],{},[60,6436,143],{},[64,6438,146],{"encoding":66}," (or a plane/hyperplane equation for multiple\nfeatures). If the real data follows a more complex pattern, the model will\nproduce inaccurate results.",[6401,6441,6443,6448],{"value":6442},"item-2",[6405,6444,6445],{"v-slot:question":6407},[13,6446,6447],{},"Why is the squared difference used in the loss function instead of the absolute value?",[6405,6449,6450],{"v-slot:answer":6407},[13,6451,6452],{},"The absolute value is mathematically inconvenient — it has no derivative\nat zero, which prevents using standard optimization methods. The squared\ndifference has a derivative everywhere and serves the same purpose: the\nlarger the error, the larger the function's value. Additionally, the\nsquare penalizes large errors more heavily, which is often useful in\npractice.",[6401,6454,6456,6461],{"value":6455},"item-3",[6405,6457,6458],{"v-slot:question":6407},[13,6459,6460],{},"What is the least squares method?",[6405,6462,6463],{"v-slot:answer":6407},[13,6464,6465],{},"It's a way of fitting model parameters by minimizing the sum of squared\ndifferences between the correct answers and the model's predictions.\nGeometrically, it means drawing a straight line (or plane) so that the\ntotal distance from it to all data points is as small as possible.",[6401,6467,6469,6474],{"value":6468},"item-4",[6405,6470,6471],{"v-slot:question":6407},[13,6472,6473],{},"Why do we set the derivatives equal to zero?",[6405,6475,6476],{"v-slot:answer":6407},[13,6477,6478],{},"From school mathematics, we know that the extremum (minimum or maximum) of\na function is found where the derivative equals zero. The loss function of\nlinear regression is a paraboloid — a surface with a single minimum.\nTherefore, the point where the derivative equals zero is guaranteed to\ngive us a minimum, not a maximum.",[6401,6480,6482,6487],{"value":6481},"item-5",[6405,6483,6484],{"v-slot:question":6407},[13,6485,6486],{},"How does linear regression work with multiple input features?",[6405,6488,6489],{"v-slot:answer":6407},[13,6490,6491,6492,6515,6516,6584],{},"The principle remains the same, only the formula expands. Instead of\n",[45,6493,6495],{"className":6494},[48],[50,6496,6497],{"xmlns":52},[54,6498,6499,6513],{},[57,6500,6501,6503,6505,6507,6509,6511],{},[60,6502,102],{},[104,6504,131],{},[60,6506,134],{},[60,6508,137],{},[104,6510,140],{},[60,6512,143],{},[64,6514,146],{"encoding":66},", we get\n",[45,6517,6519],{"className":6518},[48],[50,6520,6521],{"xmlns":52},[54,6522,6523,6581],{},[57,6524,6525,6531,6533,6539,6545,6547,6553,6559,6561,6563,6565,6571,6577,6579],{},[97,6526,6527,6529],{"accent":99},[60,6528,102],{},[104,6530,106],{"stretchy":99},[104,6532,131],{},[282,6534,6535,6537],{},[60,6536,134],{},[287,6538,289],{},[282,6540,6541,6543],{},[60,6542,137],{},[287,6544,289],{},[104,6546,140],{},[282,6548,6549,6551],{},[60,6550,134],{},[287,6552,303],{},[282,6554,6555,6557],{},[60,6556,137],{},[287,6558,303],{},[104,6560,140],{},[104,6562,506],{},[104,6564,140],{},[282,6566,6567,6569],{},[60,6568,134],{},[60,6570,455],{},[282,6572,6573,6575],{},[60,6574,137],{},[60,6576,455],{},[104,6578,140],{},[60,6580,143],{},[64,6582,6583],{"encoding":66},"\\widehat{y} = k_1 x_1 + k_2 x_2 + \\ldots + k_m x_m + b",", where each\nfeature has its own coefficient. For two features, the model builds a\nplane; for more features — a hyperplane. The training process is\nanalogous: compute derivatives with respect to each coefficient, set them\nequal to zero, and solve the resulting system of equations.",[6401,6586,6588,6593],{"value":6587},"item-6",[6405,6589,6590],{"v-slot:question":6407},[13,6591,6592],{},"Why is linear regression rarely used?",[6405,6594,6595],{"v-slot:answer":6407},[13,6596,6597],{},"The main limitation is that the model can only describe linear\nrelationships well. In real-world data, the connection between features\nand the target is often nonlinear, and a straight line (or plane) won't\nbe able to capture it. That's why linear regression is used only where\nthe relationship is indeed close to linear, and always after verifying\nthe data. But in such cases, it works significantly faster than more\ncomplex models.",[6401,6599,6601,6606],{"value":6600},"item-7",[6405,6602,6603],{"v-slot:question":6407},[13,6604,6605],{},"What is a vector in the input data?",[6405,6607,6608],{"v-slot:answer":6407},[13,6609,6610,6611,6641,6642,6660,6661,6679],{},"A vector is simply an ordered set of numbers. For example, if we're\ndetermining a person's weight from their height and waist circumference,\nthe input data consists of two numbers: ",[45,6612,6614],{"className":6613},[48],[50,6615,6616],{"xmlns":52},[54,6617,6618,6638],{},[57,6619,6620,6622,6628,6630,6636],{},[104,6621,280],{"stretchy":1271},[282,6623,6624,6626],{},[60,6625,137],{},[287,6627,289],{},[104,6629,292],{"separator":99},[282,6631,6632,6634],{},[60,6633,137],{},[287,6635,303],{},[104,6637,306],{"stretchy":1271},[64,6639,6640],{"encoding":66},"[x_1, x_2]",", where ",[45,6643,6645],{"className":6644},[48],[50,6646,6647],{"xmlns":52},[54,6648,6649,6657],{},[57,6650,6651],{},[282,6652,6653,6655],{},[60,6654,137],{},[287,6656,289],{},[64,6658,6659],{"encoding":66},"x_1"," is\nheight and ",[45,6662,6664],{"className":6663},[48],[50,6665,6666],{"xmlns":52},[54,6667,6668,6676],{},[57,6669,6670],{},[282,6671,6672,6674],{},[60,6673,137],{},[287,6675,303],{},[64,6677,6678],{"encoding":66},"x_2"," is waist circumference. This is a two-element vector.\nThe more features we use, the more elements the vector has and the more\ncoefficients we need to find during training.",[6401,6681,6683,6702],{"value":6682},"item-8",[6405,6684,6685],{"v-slot:question":6407},[13,6686,6687,6688,6701],{},"Why do we need the variable ",[45,6689,6691],{"className":6690},[48],[50,6692,6693],{"xmlns":52},[54,6694,6695,6699],{},[57,6696,6697],{},[60,6698,143],{},[64,6700,143],{"encoding":66}," (the intercept)?",[6405,6703,6704],{"v-slot:answer":6407},[13,6705,6706,6707,6720,6721,6745,6746,6768,6769,6782],{},"Without ",[45,6708,6710],{"className":6709},[48],[50,6711,6712],{"xmlns":52},[54,6713,6714,6718],{},[57,6715,6716],{},[60,6717,143],{},[64,6719,143],{"encoding":66},", the formula would look like ",[45,6722,6724],{"className":6723},[48],[50,6725,6726],{"xmlns":52},[54,6727,6728,6742],{},[57,6729,6730,6736,6738,6740],{},[97,6731,6732,6734],{"accent":99},[60,6733,102],{},[104,6735,106],{"stretchy":99},[104,6737,131],{},[60,6739,134],{},[60,6741,137],{},[64,6743,6744],{"encoding":66},"\\widehat{y} = kx",", meaning the\nline would always pass through the origin. This severely limits the model\n— not all relationships pass through the point ",[45,6747,6749],{"className":6748},[48],[50,6750,6751],{"xmlns":52},[54,6752,6753,6765],{},[57,6754,6755,6757,6759,6761,6763],{},[104,6756,803],{"stretchy":1271},[287,6758,780],{},[104,6760,292],{"separator":99},[287,6762,780],{},[104,6764,816],{"stretchy":1271},[64,6766,6767],{"encoding":66},"(0, 0)",". The coefficient\n",[45,6770,6772],{"className":6771},[48],[50,6773,6774],{"xmlns":52},[54,6775,6776,6780],{},[57,6777,6778],{},[60,6779,143],{},[64,6781,143],{"encoding":66}," allows shifting the line up or down, making the model much more\nflexible and accurate.",{"title":6407,"searchDepth":6784,"depth":6784,"links":6785},2,[6786,6787,6788,6789,6790,6791,6792],{"id":39,"depth":6784,"text":40},{"id":622,"depth":6784,"text":623},{"id":1084,"depth":6784,"text":1085},{"id":3794,"depth":6784,"text":3795},{"id":4384,"depth":6784,"text":4385},{"id":4665,"depth":6784,"text":4666},{"id":4721,"depth":6784,"text":4722},"mdx",{"readTime":6795,"image":6796,"date":6797,"tags":6798,"authors":6801},"15 minutes","/img/blog/ml-basic-2/preview.png","2026-04-17",[6799,6800],"Artificial Intelligence","Machine Learning",[6802],"vgorash",true,{"title":6805,"description":6806},"Linear Regression as Simple as It Gets","In this article, we will look at one of the simplest machine learning algorithms, namely — linear regression.","en/blog/ml-basic-2","lYm0WylilM3v3f8b8FYo8Y2Ya1Uf8mO4B1VK2tgqkxg",1777111174026]