Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

from Chapter 2 of Spark Guide 0. Construct your own schema for the flight data and use it to read in the flight data 1.

from Chapter 2 of Spark Guide

0. Construct your own schema for the flight data and use it to read in the flight data

1. Read in the Flight data ( summary-2015.csv) in DataBricks ( or IntelliJ)

1.5 Rename the DEST_... column and the ORIG_ ... column to names you like

2. Sort the DataFrame on the count column and output the first 10 rows

3. Construct a case class and convert the DataFrame to a Dataset

4.Convert your Dataset to a SQL table and use SQL to select the columns and print out the top 10,

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Database Design Application Development And Administration

Authors: Michael V. Mannino

4th Edition

0615231047, 978-0615231044

More Books

Students also viewed these Databases questions

Question

How do Data Types perform data validation?

Answered: 1 week ago

Question

How does Referential Integrity work?

Answered: 1 week ago