Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Here we will give an illustrative example of a weak learner for a simple concept class. Let the domain be the real line, R ,

Here we will give an illustrative example of a weak learner for a simple concept class. Let the domain be the real line, R, and let C refer to the concept class of 3-piece classifiers, which are functions of the following form: for 01<02 and be{-1,1}, h81,62,6(x) is b if x [01,02] and b otherwise. In other words, they take a certain Boolean value inside a certain interval and the opposite value everywhere else. For example, h10,20,1(x) would be +1 on [10,20), and -1 everywhere else. Let H refer to the simpler class of decision stumps, i.e. functions ho,6 such that h(2) is b for all x <0 and -b otherwise. (a) Show formally that for any distribution on R (assume finite support, for simplicity; i.e., assume the distribution is bounded within (-B, B] for some large B) and any unknown labeling function ce C that is a 3-piece classifier, there exists a decision stump h E H that has error at most 1/3, i.e. P[h(x)+ c(x)]<1/3.(b) Describe a simple, efficient procedure for finding a decision stump that minimizes error with respect to a finite training set of size m. Such a procedure is called an empirical risk minimizer (ERM).(c) Give a short intuitive explanation for why we should expect that we can easily pick m sufficiently large that the training error is a good approximation of the true error, i.e. why we can ensure generalization.

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Data And Databases

Authors: Jeff Mapua

1st Edition

1978502257, 978-1978502253

More Books

Students also viewed these Databases questions