Q:

Data.table - efficient evaluation of expression in "j" during group by

Is there a way of using / evaluating a string expression other than eval(parse(text = ...)) ? The example below works fine, but it is quite literally a thousand times slower than "regular" code.
From what I can tell, eval(parse()) works fine(time-wise) in both the i and the j terms unless by in used. I used verbose=TRUE and GForce is not used when using an expression, so I guess the question is how to use both GForce and the expression.
library(data.table)

N = 10**5
DT = data.table(x1 = sample(1:1000, size = N , replace = TRUE),
                x2 = sample(1:500, size = N , replace = TRUE),
                y1 = runif(N,1,10),
                y2 = runif(N,0,1))

system.time({
  DT_agg = DT[, .(sum_y1 = sum(y1)), by = c('x1','x2'), verbose=TRUE]
})
# Making each group and running j (GForce TRUE)
# user  system elapsed 
# 0.02    0.00    0.02 

expr = "sum_y1 = sum(y1)"
system.time({
  DT_agg = DT[, .(eval(parse(text = expr))), by = c('x1','x2'), verbose=TRUE]
})
# Making each group and running j (GForce FALSE)
# user  system elapsed 
# 27.72    0.00   28.11 

A:

the question is how to use both GForce and the expression.

If you (1) put the entire j argument in the expression and (2) parse it beforehand, it works:
> expr2 = quote(.(sum_y1 = sum(y1)))
> DT[, eval(expr2), by=c("x1", "x2"), verbose=TRUE]
Detected that j uses these columns: y1 
Finding groups using forderv ... 0.000sec 
Finding group sizes from the positions (can be avoided to save RAM) ... 0.020sec 
Getting back original order ... 0.000sec 
lapply optimization is on, j unchanged as 'list(sum(y1))'
GForce optimized j to 'list(gsum(y1))'
Making each group and running j (GForce TRUE) ... 0.010sec 
        x1  x2   sum_y1
    1: 377 368 1.293758
    2: 233 276 1.613304
    3: 190  97 3.432189
    4: 200 373 3.573958
    5: 924 345 5.535074
   ---                 
90538: 316 155 5.067798
90539: 960 180 5.788466
90540: 777 466 9.949981
90541: 520  43 3.815545
90542: 977 498 3.839360

