성능 튜닝 할 때 도움이 될 것 같아서 가져왔다.
https://spark.apache.org/docs/latest/tuning.html
https://tech.channable.com/posts/2018-04-10-debugging-a-long-running-apache-spark-application.html
https://nephtyws.github.io/data/spark-optimization-part-1/
https://unraveldata.com/common-reasons-spark-applications-slow-fail-part-1/
https://unraveldata.com/common-failures-slowdowns-part-ii/
https://blog.scottlogic.com/2018/03/14/apache-spark-question-everything.html
https://spark.apache.org/docs/latest/configuration.html
https://www.slideshare.net/jcmia1/apache-spark-20-tuning-guide
https://docs.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_running_spark_on_yarn.html
https://m.blog.naver.com/gyrbsdl18/220880041737
https://www.linkedin.com/pulse/tune-spark-jobs-2-chaaranpall-lambba
https://databricks.com/blog/2015/05/28/tuning-java-garbage-collection-for-spark-applications.html
http://kysepark.blogspot.com/2016/04/how-to-tune-your-apache-spark-jobs-part.html
https://medium.com/parrot-prediction/partitioning-in-apache-spark-8134ad840b0
https://tomining.tistory.com/85
'Spark' 카테고리의 다른 글
[Spark Streaming] Yahoo Benchmark github링크 (0) | 2019.12.11 |
---|---|
[Scala] map vs flatmap 차이 (0) | 2019.12.09 |
[Spark] Required executor memory (2048), overhead (384 MB), and PySpark memory (0 MB) is above the max threshold (2048 MB) of this cluster! 에러 원인 파악 및 해결 (0) | 2019.11.08 |
[Spark] mongo-spark connector 로 aggregation 쿼리 보내는 예제 코드 (0) | 2019.11.06 |
[Spark] Resource 제한하는 방법 (0) | 2019.11.05 |