Dataset A: This dataset contains input/output data generated from a noisy polynomial model: y = f(x)...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
Dataset A: This dataset contains input/output data generated from a noisy polynomial model: y = f(x) + , where x 1 = [2,2] and K f(x) = m(x,a) = px(x; a):= [: m=0 xmam+1. Here, K 1 denotes the polynomial order and a RK+ represents the parameter vector. The dataset consists of two files: SYNDTR.csv, which contains 200 training data points, and SYNDTE.csv, which contains 1800 testing data points. Dataset B: Each file has three columns: The first column represents the input (denoted as TRI for SYNDTR and TEI for SYNDTE). The second column signifies the output under low noise conditions (denoted as TROL for SYNDTR and TEOL for SYNDTE). The third column signifies the output under high noise conditions (denoted as TROH for SYNDTR and TEOH for SYNDTE). This dataset contains real-world data related to fish measurements. Specifically, the input x is 2-dimensional, where x signifies fish width and x signifies fish length. The output y is the fish weight. The dataset is divided into two files: FISHDTR.csv, which contains the training data, and FISHDTE.csv, which contains the testing data. Each file consists of three columns: "width," "length," and "weight." "Width" and "length" serve as the input features (denoted as TRI for FISHDTR and TEI for FISHDTE), while "weight" is the output (denoted as TRO for FISHDTR and TEO for FISHDTE). III. Problems Problem 1 (20%) This problem focuses on Dataset A, initially considering the low-noise (LN) scenario. Assume that you are aware that m is an M-order polynomial, denoted p, for some M. However, the exact order is unknown. Your objective is to train such that f(x) = PM(x, ) approximates the true f(x), utilizing the available training examples for regression. 1. Implement in Python: Create a function "my_model_train" that takes TRI, TROL, and M as input parameters and minimizes train-MSE to return the trained parameters . Your function should employ SVD-based single-shot LS solution. Do not use any prebuilt poly-fit functions. Implement in Python: Create a function "my_model_test" that accepts , TEI, and TEOL as inputs and calculates and returns test-MSE. 2. Implement in Python: Conduct an experiment using the aforementioned functions to compute train-MSE and test-MSE for the optimized with M = 0,1,...,10. In Fig. 1 of your report plot train-MSE (red curve) and test-MSE (blue curve) vs. M = 0,1,..., 10. Discuss how train-MSE and test- MSE vary across M in Fig. 1. 3. Implement in Python: Repeat steps 2-6 for the high-noise (HN) scenario, using the data sets TRI, TEI, TROH, and TEOH. In Fig. 2 of your report, plot train-MSE (red curve) and test-MSE (blue curve) against M. Discuss how train-MSE and test-MSE vary across M in Fig. 2 and compare these observations with those in Fig. 1. 4. Implement in Python: Repeat steps 2-8, this time limiting the training set to the first 180 data points. Create corresponding Figs. 3 and 4 and discuss the results, comparing Figs. 2 and 3 with Figs. 1 and 2. Problem 2 (20%) - Feature/Input/Output Correlations This problem is based on Dataset B. 1. Implement in Python: Utilize TRI and TRO to compute matrix C. For each (i,j) {1,2}, C(i,j) should contain the absolute value of the correlation coefficient between input features i and j. Additionally, C(i, 3) = C(3,i) should contain the absolute value of the correlation coefficient between 3 input feature i and the output. Present matrix C. Discuss the observed correlations and identify which input feature appears to be most strongly correlated with the output. 2. Implement in Python: In Fig. 5 of your report, scatter-plot weight vs width for the training data. Then, in Fig. 6 of your report, scatter- plot weight vs length for the training data. Discuss your observations based on Figs. 5 and 6. FISHDTE.csv - Notepad File Edit Format View Help width, length, weight 3.8352,22.0,169.0 1.2558,12.1,12.2 6.018,30.5,514.0 4.3056, 24.0, 290.0 3.1234,16.8,78.0 2.673,16.3,90.0 3.977,36.0,345.0 2.9415,15.7,70.0 1.408,7.5,5.9 3.555,19.0,110.0 5.1373,25.2,300.0 3.723,22.0,225.0 6.7473,37.4,975.0 5.1338,30.9,610.0 7.1064,36.9,850.0 3.525,20.0,120.0 2.9181,17.5,120.0 3.6636,21.2,200.0 4.02,23.2,242.0 3.825, 22.0,145.0 6.5736,33.7,800.0 3.3957,19.1,110.0 5.5695,31.3,575.0 7.48,59.0,1650.0 7.4624,37.0,1015.0 5.8515,32.7,714.0 4.69,27.6.390.0 FISHDTR.csv- Notepad File Edit Format View Help width, length, weight 2.268,12.9,40.0 4.1272, 22.1,200.0 4.335,25.4,265.0 1.38,10.4,9.7 5.376,44.8,770.0 3.2943,19.4,120.0 2.9044,17.5,78.0 6.8684,39.0,1100.0 2.6316,15.0,51.5 5.2785,26.8,500.0 3.3756,30.0,200.0 3.3516,19.0,0.0 3.624,20.5,150.0 4.896,42.0,500.0 3.6835,23.0,180.0 6.7408,34.0,700.0 2.0672,13.2,19.7 3.525,20.0,130.0 1.16,10.0,7.5 5.7276,31.0,650.0 7.7957,32.5,840.0 5.1042,28.4,475.0 5.3704, 31.4,685.0 6.3705,38.0,950.0 4.7736,25.0,272.0 6.1306,31.8,680.0 3.534.19.3.130.0 SYNDTR.csv - Notepad File Edit Format View Help TRI, TROL, TROH -2.0,0.7797801013578294, 3.110129736916998 -1.979899497487437,0.7334900940715499,3.522017571464012 -1.9597989949748744,0.6873845575640217,-9.401158212267045 -1.9396984924623115,0.6643223335102355, -3.7010739324007367 -1.9195979899497488,0.6045791321175524,-0.9158959772170008 -1.899497487437186,0.5863989332557281,3.837844681297914 -1.879396984924623,0.539376609695645,5.477940776987349 -1.8592964824120604, 0.47669480059185687,-2.588725395476113 -1.8391959798994975,0.46761814899616266,1.9280784320432742 -1.8190954773869348,0.4344378076039004, -1.4545775317712915 -1.7989949748743719,0.38993604832884404,7.429099183859774 -1.778894472361809,0.36941409556698585,1.5680393250941669 -1.7587939698492463,0.3247242149039432,1.7626066596825398 -1.7386934673366834,0.30337611332530134,1.9002791800822811 -1.7185929648241207,0.27695382931911905,9.63758375231981 -1.6984924623115578, 0.2417380470333166,5.159201515070963 -1.678391959798995,0.21266488428847954,2.971023707626178 -1.6582914572864322,0.18984121550654917,1.529028142347971 -1.6381909547738693,0.19995888997041456,7.4287738552470275 -1.6180904522613067,0.14080836239678757,3.674814998965255 -1.5979899497487438,0.1448777350804104, -4.205201611525807 -1.5778894472361809,0.09931189882651313,2.718938409054841 -1.557788944723618, 0.0849791192817739, -3.7120414889977074 -1.5376884422110553,0.07223210822406867,0.8380130468712469 -1.5175879396984926,0.05270961076492344,0.4786649558186202 -1.4974874371859297,0.018114822915554447,-1.4007347663089225 -1.4773869346733668,0.008053188708364923.9.606248271593234 SYNDTE.csv - Notepad File Edit Format View Help TEI, TEOL, TEOH -2.0,0.7673065575965415,0.1235565095361627 -1.9977765425236242,0.7943153591932017,1.7326977723474648 -1.9955530850472485,0.7555639265809324,1.6731494031235508 -1.9933296275708727,0.7614462415560284,1.0054139405737343 -1.991106170094497,0.7606488308433215,4.039788952036721 -1.9888827126181212,0.761312626856185, -0.29215073589232987 -1.9866592551417455,0.7489044855478995, -2.182250446781628 -1.9844357976653697,0.7570355170544998,1.7942388306922106 -1.982212340188994,0.7416562199777219,2.8470072452199324 -1.9799888827126182,0.7582756085459741, 4.074939868993058 -1.9777654252362424,0.7395068145658071,2.424837575239192 -1.9755419677598667,0.7366429368356119,10.821674885305438 -1.973318510283491,0.7241662664281812,4.508093019131171 -1.9710950528071152,0.7231835861240143,2.0058279489547886 -1.9688715953307394,0.7120192568545678,3.51913158306964 -1.9666481378543634,0.719804542660765, -7.482037309227453 -1.9644246803779877,0.7044766896317838, -9.241684444324129 -1.962201222901612,0.6948753549479644,0.3376946557104504 -1.9599777654252362,0.6957621694749753,0.10309533071685884 -1.9577543079488604,0.6861108321395015, 5.149506650813418 -1.9555308504724846,0.6813029142267479,0.028858605978796303 -1.953307392996109,0.6893520984180296,3.1650418604655735 -1.9510839355197331,0.6855969508171061,2.1792827212201322 -1.9488604780433574,0.6774649848763484,2.0721530158573676 -1.9466370205669816,0.662126569361612,3.810070166479305 -1.9444135630906059, 0.6511433187121868,2.1129793296285344 -1.9421901056142301,0.6570036100967817.1.2526295448362181 Dataset A: This dataset contains input/output data generated from a noisy polynomial model: y = f(x) + , where x 1 = [2,2] and K f(x) = m(x,a) = px(x; a):= [: m=0 xmam+1. Here, K 1 denotes the polynomial order and a RK+ represents the parameter vector. The dataset consists of two files: SYNDTR.csv, which contains 200 training data points, and SYNDTE.csv, which contains 1800 testing data points. Dataset B: Each file has three columns: The first column represents the input (denoted as TRI for SYNDTR and TEI for SYNDTE). The second column signifies the output under low noise conditions (denoted as TROL for SYNDTR and TEOL for SYNDTE). The third column signifies the output under high noise conditions (denoted as TROH for SYNDTR and TEOH for SYNDTE). This dataset contains real-world data related to fish measurements. Specifically, the input x is 2-dimensional, where x signifies fish width and x signifies fish length. The output y is the fish weight. The dataset is divided into two files: FISHDTR.csv, which contains the training data, and FISHDTE.csv, which contains the testing data. Each file consists of three columns: "width," "length," and "weight." "Width" and "length" serve as the input features (denoted as TRI for FISHDTR and TEI for FISHDTE), while "weight" is the output (denoted as TRO for FISHDTR and TEO for FISHDTE). III. Problems Problem 1 (20%) This problem focuses on Dataset A, initially considering the low-noise (LN) scenario. Assume that you are aware that m is an M-order polynomial, denoted p, for some M. However, the exact order is unknown. Your objective is to train such that f(x) = PM(x, ) approximates the true f(x), utilizing the available training examples for regression. 1. Implement in Python: Create a function "my_model_train" that takes TRI, TROL, and M as input parameters and minimizes train-MSE to return the trained parameters . Your function should employ SVD-based single-shot LS solution. Do not use any prebuilt poly-fit functions. Implement in Python: Create a function "my_model_test" that accepts , TEI, and TEOL as inputs and calculates and returns test-MSE. 2. Implement in Python: Conduct an experiment using the aforementioned functions to compute train-MSE and test-MSE for the optimized with M = 0,1,...,10. In Fig. 1 of your report plot train-MSE (red curve) and test-MSE (blue curve) vs. M = 0,1,..., 10. Discuss how train-MSE and test- MSE vary across M in Fig. 1. 3. Implement in Python: Repeat steps 2-6 for the high-noise (HN) scenario, using the data sets TRI, TEI, TROH, and TEOH. In Fig. 2 of your report, plot train-MSE (red curve) and test-MSE (blue curve) against M. Discuss how train-MSE and test-MSE vary across M in Fig. 2 and compare these observations with those in Fig. 1. 4. Implement in Python: Repeat steps 2-8, this time limiting the training set to the first 180 data points. Create corresponding Figs. 3 and 4 and discuss the results, comparing Figs. 2 and 3 with Figs. 1 and 2. Problem 2 (20%) - Feature/Input/Output Correlations This problem is based on Dataset B. 1. Implement in Python: Utilize TRI and TRO to compute matrix C. For each (i,j) {1,2}, C(i,j) should contain the absolute value of the correlation coefficient between input features i and j. Additionally, C(i, 3) = C(3,i) should contain the absolute value of the correlation coefficient between 3 input feature i and the output. Present matrix C. Discuss the observed correlations and identify which input feature appears to be most strongly correlated with the output. 2. Implement in Python: In Fig. 5 of your report, scatter-plot weight vs width for the training data. Then, in Fig. 6 of your report, scatter- plot weight vs length for the training data. Discuss your observations based on Figs. 5 and 6. FISHDTE.csv - Notepad File Edit Format View Help width, length, weight 3.8352,22.0,169.0 1.2558,12.1,12.2 6.018,30.5,514.0 4.3056, 24.0, 290.0 3.1234,16.8,78.0 2.673,16.3,90.0 3.977,36.0,345.0 2.9415,15.7,70.0 1.408,7.5,5.9 3.555,19.0,110.0 5.1373,25.2,300.0 3.723,22.0,225.0 6.7473,37.4,975.0 5.1338,30.9,610.0 7.1064,36.9,850.0 3.525,20.0,120.0 2.9181,17.5,120.0 3.6636,21.2,200.0 4.02,23.2,242.0 3.825, 22.0,145.0 6.5736,33.7,800.0 3.3957,19.1,110.0 5.5695,31.3,575.0 7.48,59.0,1650.0 7.4624,37.0,1015.0 5.8515,32.7,714.0 4.69,27.6.390.0 FISHDTR.csv- Notepad File Edit Format View Help width, length, weight 2.268,12.9,40.0 4.1272, 22.1,200.0 4.335,25.4,265.0 1.38,10.4,9.7 5.376,44.8,770.0 3.2943,19.4,120.0 2.9044,17.5,78.0 6.8684,39.0,1100.0 2.6316,15.0,51.5 5.2785,26.8,500.0 3.3756,30.0,200.0 3.3516,19.0,0.0 3.624,20.5,150.0 4.896,42.0,500.0 3.6835,23.0,180.0 6.7408,34.0,700.0 2.0672,13.2,19.7 3.525,20.0,130.0 1.16,10.0,7.5 5.7276,31.0,650.0 7.7957,32.5,840.0 5.1042,28.4,475.0 5.3704, 31.4,685.0 6.3705,38.0,950.0 4.7736,25.0,272.0 6.1306,31.8,680.0 3.534.19.3.130.0 SYNDTR.csv - Notepad File Edit Format View Help TRI, TROL, TROH -2.0,0.7797801013578294, 3.110129736916998 -1.979899497487437,0.7334900940715499,3.522017571464012 -1.9597989949748744,0.6873845575640217,-9.401158212267045 -1.9396984924623115,0.6643223335102355, -3.7010739324007367 -1.9195979899497488,0.6045791321175524,-0.9158959772170008 -1.899497487437186,0.5863989332557281,3.837844681297914 -1.879396984924623,0.539376609695645,5.477940776987349 -1.8592964824120604, 0.47669480059185687,-2.588725395476113 -1.8391959798994975,0.46761814899616266,1.9280784320432742 -1.8190954773869348,0.4344378076039004, -1.4545775317712915 -1.7989949748743719,0.38993604832884404,7.429099183859774 -1.778894472361809,0.36941409556698585,1.5680393250941669 -1.7587939698492463,0.3247242149039432,1.7626066596825398 -1.7386934673366834,0.30337611332530134,1.9002791800822811 -1.7185929648241207,0.27695382931911905,9.63758375231981 -1.6984924623115578, 0.2417380470333166,5.159201515070963 -1.678391959798995,0.21266488428847954,2.971023707626178 -1.6582914572864322,0.18984121550654917,1.529028142347971 -1.6381909547738693,0.19995888997041456,7.4287738552470275 -1.6180904522613067,0.14080836239678757,3.674814998965255 -1.5979899497487438,0.1448777350804104, -4.205201611525807 -1.5778894472361809,0.09931189882651313,2.718938409054841 -1.557788944723618, 0.0849791192817739, -3.7120414889977074 -1.5376884422110553,0.07223210822406867,0.8380130468712469 -1.5175879396984926,0.05270961076492344,0.4786649558186202 -1.4974874371859297,0.018114822915554447,-1.4007347663089225 -1.4773869346733668,0.008053188708364923.9.606248271593234 SYNDTE.csv - Notepad File Edit Format View Help TEI, TEOL, TEOH -2.0,0.7673065575965415,0.1235565095361627 -1.9977765425236242,0.7943153591932017,1.7326977723474648 -1.9955530850472485,0.7555639265809324,1.6731494031235508 -1.9933296275708727,0.7614462415560284,1.0054139405737343 -1.991106170094497,0.7606488308433215,4.039788952036721 -1.9888827126181212,0.761312626856185, -0.29215073589232987 -1.9866592551417455,0.7489044855478995, -2.182250446781628 -1.9844357976653697,0.7570355170544998,1.7942388306922106 -1.982212340188994,0.7416562199777219,2.8470072452199324 -1.9799888827126182,0.7582756085459741, 4.074939868993058 -1.9777654252362424,0.7395068145658071,2.424837575239192 -1.9755419677598667,0.7366429368356119,10.821674885305438 -1.973318510283491,0.7241662664281812,4.508093019131171 -1.9710950528071152,0.7231835861240143,2.0058279489547886 -1.9688715953307394,0.7120192568545678,3.51913158306964 -1.9666481378543634,0.719804542660765, -7.482037309227453 -1.9644246803779877,0.7044766896317838, -9.241684444324129 -1.962201222901612,0.6948753549479644,0.3376946557104504 -1.9599777654252362,0.6957621694749753,0.10309533071685884 -1.9577543079488604,0.6861108321395015, 5.149506650813418 -1.9555308504724846,0.6813029142267479,0.028858605978796303 -1.953307392996109,0.6893520984180296,3.1650418604655735 -1.9510839355197331,0.6855969508171061,2.1792827212201322 -1.9488604780433574,0.6774649848763484,2.0721530158573676 -1.9466370205669816,0.662126569361612,3.810070166479305 -1.9444135630906059, 0.6511433187121868,2.1129793296285344 -1.9421901056142301,0.6570036100967817.1.2526295448362181
Expert Answer:
Answer rating: 100% (QA)
Solutions Step 1 Problem 1 Step 1 Implement the mymodeltrain function in Python This function should take TRI TROL and M as input parameters and retur... View the full answer
Related Book For
Posted Date:
Students also viewed these programming questions
-
a) A 5-Stage pipeline is composed of the following stages Instruction Fetch (IF), Decode (DE), Execute (EX), Memory Access (ME) and Register Write-back (WB). Assume the pipeline does not have a...
-
Can you help me change this in c++ update your payroll program to use a C++ class. Each employee class object should hold the master file information for one employee. You can assume that the company...
-
The following additional information is available for the Dr. Ivan and Irene Incisor family from Chapters 1-5. Ivan's grandfather died and left a portfolio of municipal bonds. In 2012, they pay Ivan...
-
Use the limit definition of the definite integral with right Riemann sums and a regular partion to evaluate the following definite integrals. Use the Fundamental Theorem of Calculus to check your...
-
Use a random-number table to simulate the outcomes of tossing a quarter 25 times. Assume that the quarter is balanced (i.e., fair).
-
Penny, the owner of a store, makes a deal with Arthur, her manager, that at the end of the year, she receives two-thirds of the store's profit and he gets one-third. If Arthur is interested in...
-
Peco Athletics and its subsidiary, Stetson Footwear, engage in intercompany transactions, as follows: Peco sells merchandise to Stetson at a markup of 35% on cost. In 2015, Peco sold merchandise to...
-
Air pollution control specialists in southern California monitor the amount of ozone, carbon dioxide, and nitrogen dioxide in the air on an hourly basis. The hourly time series data exhibit...
-
what is the income statementBrothers Harry and Herman Hausyerday began operations of their machine shop (H & H Tool, Inc.) on January 1, 2020. The annual reporting period ends December 31. The tr 0...
-
A receiver for base band digital data has a threshold set at instead of zero. Rederive (9.8), (9.9), and (9.11) taking this into account. If P(+A) = P(-A) = 1/2, find E b / N 0 in decibels as a...
-
Coyote and Bird run a fireworks company and are trying to decide whether or not to purchase new equipment to streamline the production process. Coyote argues that they should use IRR as the basis for...
-
There is a T-bill futures contract expiring in 70 days being traded. The 91-day T-bill yield is 3.44%, the 161-day T-bill yield is 0.1%, and the risk-free rate is 0.05%. If the market price on the...
-
News about the recent wildfires generally talk about number of acres burned by the fires. Most people don't think in terms of acres. If an acre is defined to be 4840 square yards, then , i. How many...
-
Blake Company produces a single product. Last year, Blake's net operating income under absorption costing was P3,600 lower than under variable costing. The company sold 10,000 units during the year,...
-
Where is the best place in the UK to open a bakery and why? Explain
-
What would be Global companys cost of good sold. If the starting inventory $150,00 & ending inventory $90,000. but during the month Global company acquired/got $2,00,600 of inventory. (please show...
-
For all responses, please remember that providing your qualitative (written) rationale can be just as important as your quantitative calculations. 1. Below is Batteries for Everyone's budgeted...
-
Information graphics, also called infographics, are wildly popular, especially in online environments. Why do you think infographics continue to receive so much attention? How could infographics be...
-
Let : R R' be a ring homomorphism and let N be an ideal of R. a. Show that [N] is an ideal of [R]. b. Give an example to show that [N] need not be an ideal of R'. c. Let N' be an ideal either of...
-
Give a basis for the indicated vector space over the field. Q(2) over Q
-
Let a : Z 5 [x] Z 5 be an evaluation homomorphism as in Theorem 22.4. Use Fermat's theorem to evaluate 3 (x 231 + 3x 117 - 2x 53 + 1). Data from Theorem 22.4 (The Evaluation Homomorphisms for...
-
Henry Inc. purchased \(\$ 5,000\) of Container Corporation's \(5 \%\) bonds at par. The purchase is made on January 1 , 2020, and the investment is classified as a trading security. At June 30, 2020,...
-
Referring to information in Brief Exercise 14-18, assume that Henry Inc. sold its holdings of Container Corporation bonds on July 2, 2020, for \(\$ 4,800\). Record the sale of the debt investment,...
-
On January 1, 2020, Sharp Company purchased \(\$ 50,000\) of Sox Company \(5 \%\) bonds, at a time when the market rate was \(6 \%\). The bonds mature on December 31, 2024, and pay interest...
Study smarter with the SolutionInn App