Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Using PySpark: Convert all dates/times from GMT to Beijing time. (This requires to move dates, times and timestamps by 8 hours ahead). Use Sparks map

Using PySpark:

Convert all dates/times from GMT to Beijing time. (This requires to move dates, times and timestamps by 8 hours ahead). Use Sparks map transformation.

Note: I am using a larger dataset in a CSV file which I was unable to add the whole dataset.

"UserID Latitude Longitude AllZero Altitude_IGNORE Timestamp Date Time"
"118 46.77681667 121.7455333 0 4658.792651 39221.99999 19/05/2007 23:59:59"
"115 39.9861766 116.3662633 0 173.9 39572.99999 04/05/2008 23:59:59"
"115 40.0209133 116.4166183 0 157.5 39573.99999 05/05/2008 23:59:59"
"115 40.0177816 116.3743649 0 131.2 39575.99999 07/05/2008 23:59:59"
"115 39.9756316 116.3317099 0 196.9 39616.99999 17/06/2008 23:59:59"
"112 39.97176 116.4605366 0 124.7 39618.99999 19/06/2008 23:59:59"
"125 40.006229 116.324338 0 265 39618.99999 19/06/2008 23:59:59"
"115 40.0152199 116.4487916 0 157.5 39628.99999 29/06/2008 23:59:59"
"126 39.983065 117.154436 0 41 39667.99999 07/08/2008 23:59:59"
"103 39.9561166 116.34878 0 193.6 39687.99999 27/08/2008 23:59:59"
"115 40.0190416 116.3493299 0 144.4 39712.99999 21/09/2008 23:59:59"
"124 34.889338 92.930744 0 14888 39721.99999 30/09/2008 23:59:59"
"124 39.864845 116.368064 0 243 39726.99999 05/10/2008 23:59:59"
"122 39.968173 116.399708 0 180.3117651 39949.99999 16/05/2009 23:59:59"
"122 39.968327 116.399664 0 128.1290223 39977.99999 13/06/2009 23:59:59"
"113 40.052616 116.400658 0 107 40308.99999 10/05/2010 23:59:59"
"113 40.053353 116.398843 0 64 40310.99999 12/05/2010 23:59:59"
"100 39.97481403 116.3252331 0 824.1847178 40757.99999 02/08/2011 23:59:59"
"115 40.0140033 116.4540366 0 141.1 39580.99998 12/05/2008 23:59:58"

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Fundamentals Of Database Systems

Authors: Ramez Elmasri, Sham Navathe

4th Edition

0321122267, 978-0321122261

Students also viewed these Databases questions