A Learner's Platform
This post combines most frequently used command for spark, emr, yarn and AWS by hadoop developer. Kill Spark job: This command will kill all the running spark jobs.
|
1 |
ps aux | grep -i spark | awk {'print $2'} | xargs kill -9 |
...
Most common issues faced by spark developer and it’s solution Timeout waiting for connection from pool Caused by: com.amazon.ws.emr.hadoop.fs.shaded.org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool To resolve this...
More
