Saturday, 17 August 2013

Should I use every core when doing parallel processing in R?

Should I use every core when doing parallel processing in R?

I'm using R to convert some shapefiles. R does this using just one core of
my processor, and I want to speed it up using parallel processing. So I've
parallelized the process like this.
Given files which is a list of files to convert:
library(doMC)
registerDoMC()
foreach(f=files) %dopar% {
# Code to do the conversion
}
This works just fine and it uses 2 cores. According to the documentation
for registerDoMC(), by default that function uses half the cores detected
by the parallel package.
My question is why should I use half of the cores instead of all the
cores? (In this case, 4 cores.) By using the function
registerDoMC(detectCores()) I can use all the cores on my system. What, if
any, are the downsides to doing this?

No comments:

Post a Comment