Answered step by step
Verified Expert Solution
Question
1 Approved Answer
Resource management using YARN is a key component of Hadoop 2 architecture and it has multiple possible scheduler configurations. [Marks: 5] (a) Give at least
Resource management using YARN is a key component of Hadoop 2 architecture and it has multiple possible scheduler configurations. [Marks: 5] (a) Give at least 2 reasons why you may consider a Fair scheduling policy as opposed to a Capacity based policy ? (b) Suppose you are using a Fair scheduler with only 2 queues with each having weight=10. Now suppose one of the queues (Queuel) has a sudden surge of important jobs and it is at 90% utilization of it's fair share for more than a threshold time (say, 1 min). The second queue (Queue2) has no surge. In what way can this be handled by the scheduler ? State your assumptions and highlight any optimizations the scheduler will perform. (c) Suppose you are using the Capacity scheduler and a queue is configured with min=10% and max=40%. Given the queue can have multiple users, what is the mechanism to make sure that the max allocation for a single user is within 20% ?
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started