R devel - Nov 2015 - Exporting a method to a cluster object

Hi,

This is my first message to this list. It would be great if people here 
could help me with the following problem (self-contained code example 
below):

I have a list of matrices and would like to apply a summary function to 
the matrices in the list. The matrices represent social networks, 
therefore I need to apply some specialized summary functions provided by 
the ergm package. These summary statistics are contained in a summary 
method. I can write a function as a wrapper around this summary method 
and use lapply to apply the function to the list of matrices. This gives 
me the result I want:

       [,1] [,2] [,3] [,4] [,5]
edges   94   94   94   94   94
dsp1   227  227  227  227  227

However, when I try to parallelize this by using parLapply or parSapply 
from the parallel package, the results look weird:

        [,1]      [,2]      [,3]      [,4]      [,5]
Length "3"       "3"       "3"       "3"
"3"
Class  "formula" "formula" "formula"
"formula" "formula"
Mode   "call"    "call"    "call"   
"call"    "call"

And when I export summary.statistics, which is used inside my wrapper 
function, I get an error message:

Error in checkForRemoteErrors(val) :
   2 nodes produced errors; first error: could not find function 
"summary.statistics"

Do I have to export the summary method that is provided by the ergm 
package to the cluster object? If so, how? The following code is a 
self-contained example.

##########

library("ergm")
library("parallel")

# create list of matrices
m <- matrix(rbinom(900, 1, 0.1), nrow = 30)
l <- list(m, m, m, m, m)

# write wrapper function that computes results
fun <- function(mat) {
   s <- summary(mat ~ edges + dsp(1))
   return(s)
}

cl <- makePSOCKcluster(2)  # create cluster object

test1 <- sapply(l, fun)  # works!
test2 <- parSapply(cl, l, fun)  # problem: results look weird!

clusterExport(cl, varlist = "summary.statistics")
test3 <- parSapply(cl, l, fun)  # error!

##########

I found a solution somewhere in a forum:

clusterCall(cl, function() library("ergm"))

However, as I'd like to use the code in a package, the CRAN check tells 
me that using library calls inside a package is uncool. So I guess there 
must be another solution?

Thanks in advance for any hints.

Best,
Philip

(University of Bern)

On 17 November 2015 at 18:18, Philip Leifeld wrote:
| This is my first message to this list. It would be great if people here 
| could help me with the following problem (self-contained code example 
| below):

It is not really the list for it. Could you try r-sig-hpc (and you first need
to subscribe).

In short, load all required packages on all worker nodes. See the vignette of
the parallel package for a good start.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org