In base R, missing values are indicated using the
specific value NA
. Regular NAs could be
used with any type of vector (double, integer, character, factor, Date,
etc.).
Other statistical software have implemented ways to differentiate several types of missing values.
Stata and SAS have a system of
tagged NAs, where NA values are tagged with a letter
(from a to z). SPSS allows users to indicate that
certain non-missing values should be treated in some analysis as missing
(user NAs). The haven
package implements
tagged NAs and user NAs in order to
keep this information when importing files from Stata,
SAS or SPSS.
Tagged NAs are proper NA
values with a
tag attached to them. They can be created with tagged_na()
.
The attached tag should be a single letter, lowercase (a-z) or uppercase
(A-Z).
For most R functions, tagged NAs are just considered as regular NAs. By default, they are just printed as any other regular NA.
## [1] 1 2 3 4 5 NA NA NA
## [1] FALSE FALSE FALSE FALSE FALSE TRUE TRUE TRUE
To show/print their tags, you need to use na_tag()
,
print_tagged_na()
or format_tagged_na()
.
## [1] NA NA NA NA NA "a" "z" NA
## [1] 1 2 3 4 5 NA(a) NA(z) NA
## [1] " 1" " 2" " 3" " 4" " 5" "NA(a)" "NA(z)" " NA"
To test if a certain NA is a regular NA or a tagged NA, you should
use is_regular_na()
or is_tagged_na()
.
## [1] FALSE FALSE FALSE FALSE FALSE TRUE TRUE TRUE
## [1] FALSE FALSE FALSE FALSE FALSE TRUE TRUE FALSE
## [1] FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE
## [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE
Tagged NAs could be defined only for double vectors. If you add a tagged NA to a character vector, it will be converted into a regular NA. If you add a tagged NA to an integer vector, the vector will be converted into a double vector.
## [1] "a" "b" NA
## [1] FALSE FALSE FALSE
## Error: `x` must be a double vector
## [1] "double"
## [1] " 1" " 2" "NA(a)"
By default, functions such as base::unique()
,
base::duplicated()
, base::order()
or
base::sort()
will treat tagged NAs as the same thing as a
regular NA. You can use unique_tagged_na()
,
duplicated_tagged_na()
, order_tagged_na()
and
sort_tagged_na()
as alternatives that will treat two tagged
NAs with different tags as separate values.
## [1] 1 2 NA(a) 1 NA(z) 2 NA(a) NA
## [1] 1 2 NA(a)
## [1] 1 2 NA(a) NA(z) NA
## [1] FALSE FALSE FALSE TRUE TRUE TRUE TRUE TRUE
## [1] FALSE FALSE FALSE TRUE FALSE TRUE TRUE FALSE
## [1] 1 1 2 2 NA(a) NA(z) NA(a) NA
## [1] 1 1 2 2 NA(a) NA(a) NA(z) NA
It is possible to define value labels for tagged NAs.
x <- c(1, 0, 1, tagged_na("r"), 0, tagged_na("d"), tagged_na("z"), NA)
val_labels(x) <- c(
no = 0, yes = 1,
"don't know" = tagged_na("d"),
refusal = tagged_na("r")
)
x
## <labelled<double>[8]>
## [1] 1 0 1 NA(r) 0 NA(d) NA(z) NA
##
## Labels:
## value label
## 0 no
## 1 yes
## NA(d) don't know
## NA(r) refusal
When converting such labelled vector into factor, tagged NAs are, by default, converted into regular NAs (it is not possible to define tagged NAs with factors).
## [1] yes no yes <NA> no <NA> <NA> <NA>
## Levels: no yes
However, the option explicit_tagged_na
of
to_factor()
allows to transform tagged NAs into explicit
factor levels.
## [1] yes no yes refusal no don't know NA(z)
## [8] <NA>
## Levels: no yes don't know refusal NA(z)
## [1] [1] yes [0] no [1] yes [NA(r)] refusal
## [5] [0] no [NA(d)] don't know [NA(z)] NA(z) <NA>
## Levels: [0] no [1] yes [NA(d)] don't know [NA(r)] refusal [NA(z)] NA(z)
Tagged NAs can be converted into user NAs with
tagged_na_to_user_na()
.
## <labelled_spss<double>[8]>
## [1] 1 0 1 3 0 2 4 NA
## Missing range: [2, 4]
##
## Labels:
## value label
## 0 no
## 1 yes
## 2 don't know
## 3 refusal
## 4 NA(z)
## <labelled_spss<double>[8]>
## [1] 1 0 1 11 0 10 12 NA
## Missing range: [10, 12]
##
## Labels:
## value label
## 0 no
## 1 yes
## 10 don't know
## 11 refusal
## 12 NA(z)
Use tagged_na_to_regular_na()
to convert tagged NAs into
regular NAs.
## <labelled<double>[8]>
## [1] 1 0 1 NA 0 NA NA NA
##
## Labels:
## value label
## 0 no
## 1 yes
## [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
haven
introduced an haven_labelled_spss
class to deal with user defined missing values in a similar way as
SPSS. In such case, additional attributes will be used
to indicate with values should be considered as missing, but such values
will not be stored as internal NA
values. You should note
that most R function will not take this information into account.
Therefore, you will have to convert missing values into NA
if required before analysis. These defined missing values could co-exist
with internal NA
values.
User NAs could be created directly with labelled_spss()
.
You can also manipulate them with na_values()
and
na_range()
.
## <labelled<double>[8]>
## [1] 1 2 3 9 1 3 2 NA
##
## Labels:
## value label
## 1 yes
## 3 no
## 9 don't know
## <labelled_spss<double>[8]>
## [1] 1 2 3 9 1 3 2 NA
## Missing values: 9
##
## Labels:
## value label
## 1 yes
## 3 no
## 9 don't know
## <labelled<double>[8]>
## [1] 1 2 3 9 1 3 2 NA
##
## Labels:
## value label
## 1 yes
## 3 no
## 9 don't know
## [1] 5 Inf
## <labelled_spss<double>[8]>
## [1] 1 2 3 9 1 3 2 NA
## Missing range: [5, Inf]
##
## Labels:
## value label
## 1 yes
## 3 no
## 9 don't know
NB: you cant also use set_na_range()
and
set_na_values()
for a dplyr
-like syntax.
library(dplyr)
# setting value labels and user NAs
df <- tibble(s1 = c("M", "M", "F", "F"), s2 = c(1, 1, 2, 9)) %>%
set_value_labels(s2 = c(yes = 1, no = 2)) %>%
set_na_values(s2 = 9)
df$s2
## <labelled_spss<double>[4]>
## [1] 1 1 2 9
## Missing values: 9
##
## Labels:
## value label
## 1 yes
## 2 no
## <labelled<double>[4]>
## [1] 1 1 2 9
##
## Labels:
## value label
## 1 yes
## 2 no
Note that is.na()
will return TRUE
for user
NAs. Use is_user_na()
to test if a specific value is a user
NA and is_regular_na()
to test if it is a regular NA.
## <labelled_spss<double>[8]>
## [1] 1 2 3 9 1 3 2 NA
## Missing range: [5, Inf]
##
## Labels:
## value label
## 1 yes
## 3 no
## 9 don't know
## [1] FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE
## [1] FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE
## [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE
For most R functions, user NAs values are still regular values.
## <labelled_spss<integer>[10]>
## [1] 1 2 3 4 5 11 12 13 14 15
## Missing range: [10, Inf]
##
## Labels:
## value label
## 11 dk
## 15 refused
## [1] 8
You can convert user NAs into regular NAs with
user_na_to_na()
or user_na_to_regular_na()
(both functions are identical).
## <labelled<integer>[10]>
## [1] 1 2 3 4 5 NA NA NA NA NA
## [1] 3
Alternatively, if the vector is numeric, you can convert user NAs
into tagged NAs with user_na_to_tagged_na()
.
## 'x' has been converted into a double vector.
## <labelled<double>[10]>
## [1] 1 2 3 4 5 NA(a) NA(b) NA(c) NA(d) NA(e)
##
## Labels:
## value label
## NA(a) dk
## NA(e) refused
## 'x' has been converted into a double vector.
## [1] 3
Finally, you can also remove user NAs definition without converting
these values to NA
, using
remove_user_na()
.
## <labelled<integer>[10]>
## [1] 1 2 3 4 5 11 12 13 14 15
##
## Labels:
## value label
## 11 dk
## 15 refused
## [1] 8