Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Resource management using YARN is a key component of Hadoop 2 architecture and it has multiple possible scheduler configurations. [Marks: 5] (a) Give at least

Resource management using YARN is a key component of Hadoop 2 architecture and it has multiple possible scheduler configurations. [Marks: 5] (a) Give at least 2 reasons why you may consider a Fair scheduling policy as opposed to a Capacity based policy ? (b) Suppose you are using a Fair scheduler with only 2 queues with each having weight=10. Now suppose one of the queues (Queuel) has a sudden surge of important jobs and it is at 90% utilization of it's fair share for more than a threshold time (say, 1 min). The second queue (Queue2) has no surge. In what way can this be handled by the scheduler ? State your assumptions and highlight any optimizations the scheduler will perform. (c) Suppose you are using the Capacity scheduler and a queue is configured with min=10% and max=40%. Given the queue can have multiple users, what is the mechanism to make sure that the max allocation for a single user is within 20% ?
image text in transcribed
anagement Resource t using YARN is a key component of Hadoop 2 architecture and it has multiple possible scheduler configurations. [Marks: 51 (a) Give at least 2 reasons why you may consider a Fair scheduling policy as opposed to a Capacity based policy? (b) Suppose you are using a Fair scheduler with only 2 queues with each having weight-10. Now suppose one of the queues (Queuel) has a sudden surge of important jobs and it is at 90% utilization of it's fair share for more than a threshold time (say, 1 min). The second queue (Queue2) has no surge. In what way can this be handled by the scheduler? State your assumptions and highlight any optimizations the scheduler will perform (C) Suppose you are using the Capacity scheduler and a queue is configured with min=10% and max=40%.. Given the queue can have multiple users, what is the mechanism to make sure that the max allocation for a single user is within 20%? anagement Resource t using YARN is a key component of Hadoop 2 architecture and it has multiple possible scheduler configurations. [Marks: 51 (a) Give at least 2 reasons why you may consider a Fair scheduling policy as opposed to a Capacity based policy? (b) Suppose you are using a Fair scheduler with only 2 queues with each having weight-10. Now suppose one of the queues (Queuel) has a sudden surge of important jobs and it is at 90% utilization of it's fair share for more than a threshold time (say, 1 min). The second queue (Queue2) has no surge. In what way can this be handled by the scheduler? State your assumptions and highlight any optimizations the scheduler will perform (C) Suppose you are using the Capacity scheduler and a queue is configured with min=10% and max=40%.. Given the queue can have multiple users, what is the mechanism to make sure that the max allocation for a single user is within 20%

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Databases DeMYSTiFieD

Authors: Andy Oppel

2nd Edition

0071747990, 978-0071747998

More Books

Students also viewed these Databases questions