Study English 23.08.02

 

Q. What is executor memory in spark?

A.

Executors have their own memory that is like a worktable.

- Executors have their own memory that is like a work table.

 

Spark driver gives tasks to executors and executors put related data on memory to treat the tasks.

- The Spark driver gives tasks to the executors. In turn, Executors put related data on the memory to deal with the tasks.

 

 in turn : 차례차례, 결국[결과적으로]

 

This is why spark is much faster than Hadoop MapReduce.

- This is why Spark is much faster than hadoop map reduce.

 

Memory is better than disk. Also, Much memory, faster processing.

- Memory is better than a disk and more memory equals faster processing.

 

It's okay even memoy space is smaller than data.

- It's ok if memory space is smaller than the data

 

Spark use disk spaces for exceeded data over memory size.

- because Spark uses disk space for exceeded data over memory size.

 

Sometimes data on memory can be deleted by accident but no worry, rdd remembers the data hierarchy so the deleted data will be recovered again.

- Sometimes data on memory can be deleted by accident. But not to worry, RDD remembers the data hierarchy. So the deleted data can be recovered again.

 

We need to use cache function for some data that is used for wide transformation many times.

- We need to use a cache function for some data that is used for wide transformation many times.

 

Q. Explain how columer storage increases the core speed?
A. The columnar storage has its data in order by column.
So when queries search a few columns, it's faster and better than row storage.
When the query is executed the data is inline in the physical memory.
There is no need to move the reading points here and there.
And so we can save disk space because there are no empty spaces to explain one row.
In addition, encrypt functions will be processed faster for data in columnar storage.