38 When Life is Linear 3 2.5 1 0 .5 1 0.5 2 1 0 Figu re 5.6. Rotating a triangle by 90 degrees about a point (a) and reflecting a triangle about a line (b). by the matrix 0 2 1 ー 1 0 L4240 0 1 0.8640 0 0 1 TO reflect a point about the x or y-aXlS, we multiply with the matrlces ーー 0 0 0 1 0 0 0 1 - 0 ・ 0 一 1 0 1 0 1 0 0 or respectively. More generally, to reflect about the line ax 十十 c = 0 , we multiply by the matr1X 2 ー 200 ー 20 わ 4 ー 2 わ 0 42 十わ 2 0 A reflection about the line 2x 十 2 ) , ー 1.5 = 0 is seen ⅲ Figure 5.6 (b), which corresponds to multiplying with the matr1X 0 0 This gives us all the linear algebra we need t0 rotate in 2D and tile the plane with the model bird. When tiling, one is 厄代 with such decorating decisions as a choice Of COlor and the amount Of surface area tO cover. Having tiled in 2D, let's step up a dimenslon and rotate in 3D. ー加わ 0 2

7 X Marks the Spot this book, we will think linearly.ln two dimensrons, this means a line. ln three dimenslons, we're talking about a plane. ln higher dimenslons, math helps us work with hyperplanes. A lot 0f the world isn't linear. The world, especially the natural world, often offers beautiful curves. Yet, like the horlzon, curvature, if viewed in the nght way, can Ok linear. The ability tO approximate curves with lines will be important tO many portions 0f this b00k. TO get a visual sense 0f modeling curved space with lines, consider sketching only with straight lines. HOW about drawing a portrait? l'lllay down a series 0f dots that approximate an image and then I'II draw one continuous line through all the points. l'll start and end at the same point. See Figure l. 1 for an example. Recognize the image in Figure l. 1 ? Such visual art, called TSP Art, was introduced and developed in 卩 , 4 ]. 。、 TSP" stands for 、、 traveling salesperson problem" since the underlying dots can be viewed as cities and the line segments between dOts indicate the route the salesperson will make through the cities. Such problems help minimize travel. Though the image is not the onginal portrait, the drawing is recog- nizable. The line drawing captures, in this case, visual components Of the onginalimage. Later, we will use linear phenomenon tO model sports per- formance enabling us t0 predict future play. But, let's not get t00 far ahead 0f ourselves. Most 0f this b00k will explore linear systems, essentially puzzles wrltten as equations. Let's see an example that l'll pose in the form 0f a magic tnck. 1

73 End ofthe Line We've about finished ourjourney through topics 0f linear algebra and appli- cations that can be studied with them. As we've learned tOOls, we've seen a vanety 0f ways tO apply ourlearning. The early chapters dealt extensively with applications 0f computer graphics and demonstrated the power and versatility Of even simple mathe- matical operations. Such tOOls enable the computations tO be done quickly, which can be useful and important in video games where quick calcula- tions are necessary for user-defined movement Of characters like the Viking in Figure 13.1. For example, Figure 13.1 (a) shows an underlying wire- frame model as created for movement while Figure 13.1 (b) removes its lines, enhancing the sense Of realism and three dimensional Ok Of the game. The later chapters delved more intO areas Of data mining and analytics. ln today's world the amount 0f information available and accessible about our world continually enlarges. Linear algebra is a means tO harness this potential power. We've learned some basic methods and many Others eXISt. However, it's important tO know that methods continue tO be developed, t00. And here is our last lesson. Even the field 0f linear algebra is far from linear. There are new tWiStS and turns in hOW methods are used and new techniques that are created. The field cannot be accurately rep- resented by a directed line segment. lt continues tO grow, encouraged, ⅲ part, by the many applications that derive from the usefulness 0f linear algebra. The boundanes 0f linear algebra also grow out 0f the creativity and ingenuity Of mathematicians, like you. What applications interest you and came tO mind as you read? DO new ideas present themselves as you

56 When Life is Linear Table 7.1. 0 作〃 / c GoId M ビイ記〃ⅲ〃 g 7 加肥 s ⅲ立 co 〃ホ′・ the た〃 3 700 Meter D のん Year 96 円 00 円 04 円 06 ー 908 1 9 に 円 20 Time に Ⅱ Ⅱ Ⅱ .2 川 .8 IO. 8 10.8 Year 1924 1928 1932 円 36 円 48 1952 円 56 Time 川 .6 10.8 IO. 3 10.3 10.3 10.79 川 .62 Year 1960 円 64 円 68 円 72 円 76 円 80 円 84 Time 10.32 10.06 9.95 川」 4 川 .06 川 .25 9.99 Year 1988 円 92 1996 2000 2004 2008 20 に 9.92 9.96 9.84 9.87 9.85 9.69 9.63 if the trend in the data continues, we can make predictions at how fast we might be running ⅲ 2020 or 2040. First, let's see how to model this data with a line and then how to predict the future. We wanttO find a liney = ax 十わ , wherex IS ayearandy isthe time run inthatyear. lfthe points all lie alonga line,thenwe'dhave 9.63 = 川 ( 2012 ) 十 わ , using B01t's second gold medaltime. Similarly, Tom Burke's performance leads to the equation 12 = 川い 896 ) 十わ . The two linear equations, for Bolt and Burke, correspond tO the system Of equations 2012 1 川 1896 1 We can easily solve this system and find 川 9.63 12 ー 0.0204 and わ = 50.7372. So, using only tWO times, we find the equation ー 0.0204X 十 50.7372. Let's check this on tlmes we didn't use tO find the equation. For example, consider Carl Lewis's 1984 time 0f9.99. We check the line by letting ェ = 1984. So, ー 0.0204 ( 1984 ) 十 50.7372 = 10.2636. The time doesn't match but it 10.8 10.6 10.4 10 9.8 1900 1920 1940 1960 1980 2000 11 .8 1 1 .6 10.8 10.6 10.4 102 1900 9.8 0 1920 1940 1960 1980 2000 Figure 7.1. Olympic go 旧 medaltimes in the men's 川 0 m dash (a) and a least squares fit t0 the data (b).

2 When Life is Linear Figu re 1 1. A portrait of Marilyn Monroe drawn with straight lines. Photograph courtesy Of RObert Bosch. Think of a number between 1 and 20. Double the number. Now, add 8. Next, divide by 2. Subtract your onginal number and mark the spot on the number line where this computed number lies with an x. 0 5 20 I bet I know where . r lies on the number line. lt lies at the value 4. How did ー know? ー did this by first using a varlable 必 and ) , equals the orlginal number you chose, regardless 0f its value.ln the second step, you doubled it producing 2 Adding 8 resulted in 2 十 8 , and then dividing by 2 led to ( 21 十 8 ) / 2 = 十 4. FinaIIy,I asked you to subtract that onginal number.l never knew its value but did know that it equalled so, when you subtrac い , from 十 4 , you indeed get 4. SO, my guess wasn't much ofa guess; rather it was some algebra—in fact, / ⅲ e の・ algebra since all the terms い , , 2 ) , 十 8 and so 応 h ) are linear. I never needed t0 know your orlginal number. ln fact, it wouldn't have mattered if you selected a number between ー 100 and 100 or a number with a million digits. Regardless, your 升 n 引 value would always result in 4. Solving for unknowns is a powerfultool of linear algebra. Here's a problem you may recall 什 om a math class that again looks for one unknown number. The AIpha Train, traveling 80 miles per hour (mph),leaves West- ville heading toward Eastburg, 290 miles away. SimuItaneously,

78 When Life is Linear 0 8 6 4 00 0 2 0 8 6 4 つ」 0 2 20 10 15 5 5 20 ー 10 ー 5 0 10 15 20 20 ー 15 ー 10 ー 5 0 Figu re 8.8. Finding the variance of the points in Figure 8.7 什 om the y-axis (a) and . r-axis (b). The most important point is that no other straight line has a larger varrance than the vertical one. SO, the largest vanance Of the data occurs over the x-coordinates. Therefore, the x-axis is the prlncipal component for this data. HOW dO ー know no better line exists? An eigenvector tOld me. Let's see hO 、 V. For eigenvectors tO help us, 、 first construct a matrIX 、 vhose columns contain the coordinates Ofthe points ln our data. For our example, our matr1X of points ⅲ Figure 8.7 , rounded t0 the first decimal place, is 20 17.3 10.0 0 ー 10.0 ー 17.3 ー 20 ー 17.3 ー 10.0 0 10.0 に .3 0 5.0 8.7 10 5.0 0 ー 5.0 ー 8.7 ー 10 ー 8.7 ー 5.0 The matr1X that we analyze is C = PPT which for this example, is 2400 0 600 For a diagonal matr1X the eigenvalues are the diagonal elements. The eigen- values indicate the varlance in the direction Ofthe corresponding elgenvector. SO, the largest eigenvalue is 2400 and its associated eigenvector is: 1 0 which matches what we found earlier. Now let's rotate the data by 50 degrees as seen in Figure 8.9 (a). If we again form the matrix = p p T where the points are now the rotated points seen in Figure 8.9 (a) then the eigenvalues are again 2400 and 600 but now the eigenvector associated with the largest eigenvalue is ー 0.6428 0.7660 0

34 When Life is Linear 5.3 ScaIing t0 Higher Dimensions 、 Me've been multiplying vectors, which are matrlces with only one row or one column. NOW, let's learn tO multiply tWO matrlces without such a restrlction. We need t0 know that B does not necessarlly equal 召月 . ln fact, while it might be possible t0 calculate B, BA might not be computable. We know when matnx multiplication is possible by looking at the dimenslons of the matnces ⅲ the product. げ図 is 襯 >< 〃 and 召 is 〃 >< 9 , then 〃 must equal 〃 in order t0 perform AB; when 〃 = 〃 , the resulting matnx is >< 9. The number 0f columns 0f .4 must equal the number 0f rows Of 召 to perform B. For example, if then 図 B is computable since has three columns and B has three rows. The product will be a matnx with three rows and two columns. The element in row / and column ノ 0f C, where C = 月召 , is found by computing the scalar product 0f row / 0f with the jth column of B. So, the element ⅲ the first row and first column of C equals We do this type of multiplication to calculate every element in C and find 2 17 Computing 召月 is not possible since B has two columns and 月 has three ro ′ S. 1 0 - ー 1 ス ) 4 っ 4 1 0- and B = っ ~ 1 0- ー 1 2 0 ] ー 2 ー 2 十 0 = 5.4 Escherin the Matrix Armed with the ability t0 multiply matrrces together, let's take a spin at tiling the plane. Consider the tile ⅲ Figure 5.4 constructed with ten line seg- ments. We'II call this the model bird. The ten endpoints ofthe line segments are ( 0.3036 , 0.1960 ) , ( 0.6168 , 0.2977 ) , ( 0.7128 , 0.4169 ) , ( 0.7120 , 0.1960 ) ,

which is called the 尾豆面記 0 Returning tO our first chOice Of 川 residual vector IS 58 The error in the approximation is computed by ー 0.0204 and わ = 50.7372 , the When Life is Linear 8.7480 11.1840 9.3360 9.63 12 9.99 2012 1896 1984 1 1 1 ー 0.0204 50.7372 9.63 12 9.99 ー 0.8820 0.2736 0.0588 0.0624 10.2636 12.0588 9.6924 ー 0.6540 ー 0.8160 9.63 12 9.99 Between these two choices for 襯 and わ , which is better? TO answer this, we need a mathematical sense Of what is meant by 、、 better. " this chapter, a smaller / e づ la 尾 s 夜・ ro implies a better approximation tO the data. Least-squares error equals the sum Of the squares Of the terms Of the re Si dual vector. For the residual vector ー 0.8820 ー 0.8160 ー 0.6540 the least squares error equals ( ー 0.8820 ) 2 十 ( ー 0.8160 ) 2 十 ( ー 0.6540 ) 2 = 1 .8715. For the residual vector 0.0624 0.0588 0.2736 the least squares error equals ( 0.0624 ) 2 十 ( 0.0588 ) 2 十 ( 0.2736 ) 2 = 0.0822. Since our first chOice for 川 and わ produced a lower least-squares error, the resulting line, ー 0.0204X 十 50.7372 , is a better fit to the data. ThiS iS where the process Of mimnuzatl()n enters. ()ur question is: among all choices for 川 and わ , which produces the smallest least-squares error? The least-squares error is the square of the length of the residual vector. SO, our problem can alSO be Vie 、 as choosing values Of 襯 and わ that produce the shortest residual vector.

57 Math tO the Max won't since ( 1984 , 9.99 ) , ( 2012 , 9.63 ) , and い 896 , 12 ) will never lie on the same line. But, can we produce a better estimate with another line? Maybe we could try connecting another tWO points, but this takes considerable fiddling and isn't necessary. And, how would we know when we have the best line notjust tO estimate Carl Lewis's time but everyone's? This issue leads us t0 the method oflinear least squares. Let's add Carl Lewis's time tO see, on a small system, hOW tO find such a line that fits the data. 、 Ve are now interested in the linear system 9.63 9.99 which we will denote as Mx = b. This system doesn't have a solution. げ it did, then we could find a line passing through all the points. SO, instead, we want a line that is closest t0 all the points, possibly not passing through any. Take a moment and consider the task. Ⅵで have infinitely many choices form and わ . Let's picktwo sets ofvalues. Earlier, when we used onlythe times of Tom Burke and Usain BoIt, we found 川 = ー 0.0204 and わ = 50.7372. These values correlate to a model in which the gold medal winmng time decreases by 0.0204 0f a second per year. The . v-intercept 0f the line is how fast people, again under this model, would have run 100 meters in year 0. ls this model stating that the fastest person tWO thousand years ago would have run 100 meters in 50 seconds? a sense, yes. Possibly a better way t0 0k at it, though, is that the model doesn't accurately extend back that far. Keep ln mind that our data covered Just over one hundred years. Expecting it tO accurately model phenomenon for tWO thousand years is asking t00 much. Now, let's make another choice, and set 川 = ー 0.021 and わ = 51. Remember, these values predict the values in our linear system, which are those gold medal winning times. TO see what 襯 = ー 0.021 and わ = 5 1 predict for gold medaltimes, we compute 2012 1 ー 0.021 96 ー 円 84 1 real winning times 、 vere っ 4 一 6 一 -4- 〃 7 8.7480 Ⅱ . 1840 9.3360 9.63 12 9.99

Math to the Max 59 How dO we pick among an infinite number Of choices for 川 and わ ? Somewhat amazingly, we need only t0 multiply b0th sides ofthe equation by MT which simply means we want t0 solve MTMx = M b. For our small system, this corresponds tO solving 20 に 1 1896 1 円 84 ー 2 田 2 1896 円 84 1 2012 1896 1984 1 1 1 9.63 9.99 12 which, after performing the matnx multiplications, becomes 川 6998484 54726 54726 28 or 川 = ー 0.0210 andb = 5L8033. 567962.44 290.84 Let's revisit what makes this the best line. At ズ = 20 ー 2 , it estimates the time t0 be ) , = 2012 ( ー 0.0210 ) 十 5L8033 = 9.5513. The vertical distance from the point ( 20 に , 9.63 ) t0 the line is the distance between ( 2012 , 9.5513 ) and ( 2012 , 9.63 ) , which equals9.63 ー 9.5513 = 0.0787. Forthe year 1896 , the line estimates the time as 1 L9873 , so the error is 12 ー 1 L9873 0.0 に 7. Finally, for 1984,thelinewouldreturny = 10 」 393 , and 川 .1393 ー 9.99 = 0.1493. The sum 0f the squares 0f the distances is ( 0.0787 ) 2 十 ( 0. 田 27 ) 2 十 ( 0 」 493 ) 2 = 0.0286. This line, produced using the method ofleast squares, chose the 川 and わ such that this sum as small as possible.ln this sense, it is the best fit of the data. NO other line will produce a smaller sum. What if we use all the data? This produces the line ァ = ー 0.0133X 十 36. 引 , which is graphed in Figure 7.1. N0tice how close the line is t0 vanous points. For 2012 , it predicts Usain B01t's time at 9.599 , which is only .003 seconds 0 仼 his actual time! Let's 0k at our least squares line more closely. lts slope is ー 0.0133. Remember, . denotes the year and is the time. SO, the slope predicts that every year, we expect the Olympic gold medal time t0 drop by just over a hundredth of a second. At what point would someone reach the limit Of speed? NO one has ever run the race in 2 seconds. SO, there is some limit between 2 and 9.63 seconds. The bOOk, The R 夜 : c 行 0 〃 PO ⅲな considers such a question. lt is written by J0hn Brenkus, the host 0f ESPN 's Sports Science show. Brenkus analyzes four distinct phases Of the race: reacting tO the gun, getting out Of the blocks, accelerating to top speed, and hanging on for dear ⅱ応 at the end.