A nested data frame is a data frame where one (or more) columns is a list of data frames. You can create simple nested data frames by hand:
df1 <- tibble(
g = c(1, 2, 3),
data = list(
tibble(x = 1, y = 2),
tibble(x = 4:5, y = 6:7),
tibble(x = 10)
)
)
df1
#> # A tibble: 3 × 2
#> g data
#> <dbl> <list>
#> 1 1 <tibble [1 × 2]>
#> 2 2 <tibble [2 × 2]>
#> 3 3 <tibble [1 × 1]>
(It is possible to create list-columns in regular data frames, not
just in tibbles, but it’s considerably more work because the default
behaviour of data.frame()
is to treat lists as lists of
columns.)
But more commonly you’ll create them with
tidyr::nest()
:
df2 <- tribble(
~g, ~x, ~y,
1, 1, 2,
2, 4, 6,
2, 5, 7,
3, 10, NA
)
df2 %>% nest(data = c(x, y))
#> # A tibble: 3 × 2
#> g data
#> <dbl> <list>
#> 1 1 <tibble [1 × 2]>
#> 2 2 <tibble [2 × 2]>
#> 3 3 <tibble [1 × 2]>
nest()
specifies which variables should be nested
inside; an alternative is to use dplyr::group_by()
to
describe which variables should be kept outside.
df2 %>% group_by(g) %>% nest()
#> # A tibble: 3 × 2
#> # Groups: g [3]
#> g data
#> <dbl> <list>
#> 1 1 <tibble [1 × 2]>
#> 2 2 <tibble [2 × 2]>
#> 3 3 <tibble [1 × 2]>
I think nesting is easiest to understand in connection to grouped data: each row in the output corresponds to one group in the input. We’ll see shortly this is particularly convenient when you have other per-group objects.
The opposite of nest()
is unnest()
. You
give it the name of a list-column containing data frames, and it
row-binds the data frames together, repeating the outer columns the
right number of times to line up.
Nested data is a great fit for problems where you have one of something for each group. A common place this arises is when you’re fitting multiple models.
mtcars_nested <- mtcars %>%
group_by(cyl) %>%
nest()
mtcars_nested
#> # A tibble: 3 × 2
#> # Groups: cyl [3]
#> cyl data
#> <dbl> <list>
#> 1 6 <tibble [7 × 10]>
#> 2 4 <tibble [11 × 10]>
#> 3 8 <tibble [14 × 10]>
Once you have a list of data frames, it’s very natural to produce a list of models:
mtcars_nested <- mtcars_nested %>%
mutate(model = map(data, function(df) lm(mpg ~ wt, data = df)))
mtcars_nested
#> # A tibble: 3 × 3
#> # Groups: cyl [3]
#> cyl data model
#> <dbl> <list> <list>
#> 1 6 <tibble [7 × 10]> <lm>
#> 2 4 <tibble [11 × 10]> <lm>
#> 3 8 <tibble [14 × 10]> <lm>
And then you could even produce a list of predictions:
mtcars_nested <- mtcars_nested %>%
mutate(model = map(model, predict))
mtcars_nested
#> # A tibble: 3 × 3
#> # Groups: cyl [3]
#> cyl data model
#> <dbl> <list> <list>
#> 1 6 <tibble [7 × 10]> <dbl [7]>
#> 2 4 <tibble [11 × 10]> <dbl [11]>
#> 3 8 <tibble [14 × 10]> <dbl [14]>
This workflow works particularly well in conjunction with broom, which makes it easy to
turn models into tidy data frames which can then be
unnest()
ed to get back to flat data frames. You can see a
bigger example in the broom
and dplyr vignette.