rowsum() and colsum() on a DelayedMatrix object
DelayedMatrix-rowsum.RdLike ordinary matrices in base R, DelayedMatrix objects and
derivatives support rowsum() and colsum().
Details
Note that the rowsum() and colsum() operations are not
delayed: the output matrix is realized block by block.
The automatic realization backend controls where realization
happens e.g. in memory as an ordinary matrix if not set (i.e. set to
NULL), or in an HDF5 file if set to "HDF5Array".
See ?setAutoRealizationBackend for more information
about realization backends.
Value
The object returned by the rowsum() or colsum() method
for DelayedMatrix objects will be either:
An ordinary matrix if the automatic realization backend is
NULL(the default).A DelayedMatrix object if the automatic realization backend is not
NULL. In this case, the returned DelayedMatrix object will be either pristine or made of several pristine DelayedMatrix objects bound together (viarbind()orcbind(), both are delayed operations).For example, if the automatic realization backend is
"HDF5Array", then the returned DelayedMatrix object will be either an HDF5Array object, or it will be a DelayedMatrix object made of several HDF5Array objects bound together.
See also
rowsumin base R.S4Arrays::rowsumin the S4Arrays package for therowsum()andcolsum()S4 generic functions.getAutoRealizationBackendandsetAutoRealizationBackendfor getting and setting the automatic realization backend.matrixStats-methods for DelayedMatrix row/col summarization.
DelayedMatrix-mult for DelayedMatrix multiplication and cross-product.
DelayedArray objects.
writeHDF5Arrayin the HDF5Array package for writing an array-like object to an HDF5 file and other low-level utilities to control the location of automatically created HDF5 datasets.HDF5Array objects in the HDF5Array package.
Examples
library(HDF5Array)
set.seed(123)
m0 <- matrix(runif(14400000), ncol=2250,
dimnames=list(NULL, sprintf("C%04d", 1:2250)))
M0 <- writeHDF5Array(m0, chunkdim=c(200, 250))
dimnames(M0) <- dimnames(m0)
## --- rowsum() ---
group <- sample(90, nrow(M0), replace=TRUE) # define groups of rows
rs <- rowsum(M0, group)
rs[1:5, 1:8]
#> C0001 C0002 C0003 C0004 C0005 C0006 C0007 C0008
#> 1 39.81194 38.64529 35.04483 39.88746 37.84819 39.25057 37.83251 39.93769
#> 2 35.30461 38.24337 39.28806 34.46383 32.48662 32.12644 38.96682 35.90867
#> 3 38.57349 30.68525 33.54541 39.13744 33.85439 34.76404 37.56156 35.77075
#> 4 30.32653 30.99076 30.61838 29.39803 34.32504 34.51552 29.25685 28.74806
#> 5 39.43013 44.22959 39.67990 41.22843 42.53644 41.15772 39.40156 43.35246
rs2 <- rowsum(M0, group, reorder=FALSE)
rs2[1:5, 1:8]
#> C0001 C0002 C0003 C0004 C0005 C0006 C0007 C0008
#> 21 31.82725 32.94878 34.69762 37.91506 37.78825 34.59857 36.23668 35.94139
#> 28 24.94209 24.88739 21.46321 22.51600 23.88168 23.97929 27.86087 27.48410
#> 40 29.30654 26.50739 26.29035 26.82618 31.31480 27.32449 28.99450 29.08730
#> 30 34.94425 31.75906 36.56136 36.35506 30.55664 32.52279 37.46168 32.65966
#> 66 40.90475 41.57575 33.89590 34.88499 37.81320 37.40290 40.30604 38.02634
## Let's see block processing in action:
DelayedArray:::set_verbose_block_processing(TRUE)
#> [1] FALSE
setAutoBlockSize(2e6)
#> automatic block size set to 2e+06 bytes (was 1e+08)
rs3 <- rowsum(M0, group)
#> === START walking on vertical strip 1/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 1/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 1/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 1/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 1/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 1/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 1/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 1/9]] ...
#> ok
#> === DONE walking on vertical strip 1/9 ===
#> --- START finalizing result for vertical strip 1/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 1/9 ---
#>
#> === START walking on vertical strip 2/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 2/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 2/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 2/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 2/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 2/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 2/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 2/9]] ...
#> ok
#> === DONE walking on vertical strip 2/9 ===
#> --- START finalizing result for vertical strip 2/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 2/9 ---
#>
#> === START walking on vertical strip 3/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 3/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 3/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 3/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 3/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 3/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 3/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 3/9]] ...
#> ok
#> === DONE walking on vertical strip 3/9 ===
#> --- START finalizing result for vertical strip 3/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 3/9 ---
#>
#> === START walking on vertical strip 4/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 4/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 4/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 4/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 4/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 4/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 4/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 4/9]] ...
#> ok
#> === DONE walking on vertical strip 4/9 ===
#> --- START finalizing result for vertical strip 4/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 4/9 ---
#>
#> === START walking on vertical strip 5/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 5/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 5/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 5/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 5/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 5/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 5/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 5/9]] ...
#> ok
#> === DONE walking on vertical strip 5/9 ===
#> --- START finalizing result for vertical strip 5/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 5/9 ---
#>
#> === START walking on vertical strip 6/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 6/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 6/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 6/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 6/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 6/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 6/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 6/9]] ...
#> ok
#> === DONE walking on vertical strip 6/9 ===
#> --- START finalizing result for vertical strip 6/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 6/9 ---
#>
#> === START walking on vertical strip 7/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 7/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 7/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 7/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 7/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 7/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 7/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 7/9]] ...
#> ok
#> === DONE walking on vertical strip 7/9 ===
#> --- START finalizing result for vertical strip 7/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 7/9 ---
#>
#> === START walking on vertical strip 8/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 8/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 8/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 8/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 8/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 8/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 8/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 8/9]] ...
#> ok
#> === DONE walking on vertical strip 8/9 ===
#> --- START finalizing result for vertical strip 8/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 8/9 ---
#>
#> === START walking on vertical strip 9/9 ===
#> | processing <1000 x 250> block from grid position [[1/7, 9/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[2/7, 9/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[3/7, 9/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[4/7, 9/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[5/7, 9/9]] ...
#> ok
#> | processing <1000 x 250> block from grid position [[6/7, 9/9]] ...
#> ok
#> | processing <400 x 250> block from grid position [[7/7, 9/9]] ...
#> ok
#> === DONE walking on vertical strip 9/9 ===
#> --- START finalizing result for vertical strip 9/9 ---
#> | result is a <90 x 250> matrix --> returning as-is
#> --- DONE finalizing result for vertical strip 9/9 ---
#>
#> === FINAL STEP ===
#> | cbind()'ing strip results (9 matrices) together ...
#> ok
#> === DONE ===
setAutoBlockSize()
#> automatic block size set to 1e+08 bytes (was 2e+06)
DelayedArray:::set_verbose_block_processing(FALSE)
#> [1] TRUE
## Sanity checks:
stopifnot(all.equal(rowsum(m0, group), rs))
stopifnot(all.equal(rowsum(m0, group, reorder=FALSE), rs2))
stopifnot(all.equal(rs, rs3))
## --- colsum() ---
group <- sample(30, ncol(M0), replace=TRUE) # define groups of cols
cs <- colsum(M0, group)
cs[1:5, 1:7]
#> 1 2 3 4 5 6 7
#> [1,] 30.14770 33.10845 32.13662 31.82049 42.83277 35.21581 39.50626
#> [2,] 37.03437 39.75568 34.80177 34.15911 38.56651 38.03176 45.37207
#> [3,] 31.75641 37.18876 33.97502 33.79669 40.32939 38.68158 42.41445
#> [4,] 34.77660 41.92982 32.01271 34.71390 39.53581 35.86748 41.94365
#> [5,] 36.97099 41.10320 32.83927 34.33561 40.45375 35.97122 40.52406
cs2 <- colsum(M0, group, reorder=FALSE)
cs2[1:5, 1:7]
#> 25 14 16 29 26 2 10
#> [1,] 31.94518 40.08585 37.35379 36.90610 45.70719 33.10845 33.97259
#> [2,] 32.20494 45.79939 41.59600 40.91405 43.76673 39.75568 34.21700
#> [3,] 32.29158 44.25482 39.02949 36.34091 43.82730 37.18876 37.06936
#> [4,] 35.53189 44.91650 44.32845 33.60211 41.14465 41.92982 34.57454
#> [5,] 31.25638 43.05772 45.24444 34.15502 46.25302 41.10320 30.36828
## Sanity checks:
stopifnot(all.equal(colsum(m0, group), cs))
stopifnot(all.equal(cs, t(rowsum(t(m0), group))))
stopifnot(all.equal(cs, t(rowsum(t(M0), group))))
stopifnot(all.equal(colsum(m0, group, reorder=FALSE), cs2))
stopifnot(all.equal(cs2, t(rowsum(t(m0), group, reorder=FALSE))))
stopifnot(all.equal(cs2, t(rowsum(t(M0), group, reorder=FALSE))))