Load, Save, and .rda files

A couple weeks ago I stumbled across a feature in R that I had never heard of before. The functions save(), load(), and the R file type .rda.

The .rda files allow a user to save their R data structures such as vectors, matrices, and data frames. The file is automatically compressed, with user options for additional compression. Let’s take a look.

First, we will grab one of the built-in R datasets. We can view these by calling data(). Let’s use the “Orange” dataset.

# get the Orange data
Orange
   Tree  age circumference
1     1  118            30
2     1  484            58
3     1  664            87
4     1 1004           115
5     1 1231           120
6     1 1372           142
7     1 1582           145
8     2  118            33
9     2  484            69
10    2  664           111
11    2 1004           156
12    2 1231           172
13    2 1372           203
14    2 1582           203
15    3  118            30
16    3  484            51
17    3  664            75
18    3 1004           108
19    3 1231           115
20    3 1372           139
21    3 1582           140
22    4  118            32
23    4  484            62
24    4  664           112
25    4 1004           167
26    4 1231           179
27    4 1372           209
28    4 1582           214
29    5  118            30
30    5  484            49
31    5  664            81
32    5 1004           125
33    5 1231           142
34    5 1372           174
35    5 1582           177

Next, let’s save each column individually as vectors.

# save the Orange data as vectors
count<-Orange$Tree
age<-Orange$age
circumference<-Orange$circumference

Now if we look at our variables in the RStudio environment, we can see count, age, and circumference saved there.
saved_files

Next, let’s set our R working directory, so the .rda file will save in the correct location. First we’ll use getwd() to find our current working directory, then we’ll adjust it (if needed) using setwd(). I set my working directory to a folder on the D drive.

#get and set working directory
getwd()
[1] "D:/Users"
setwd("D:/r-temp")
> getwd()
[1] "D:/r-temp"

Finally, let’s use the save() command to save our 3 vectors to an .rda file. The “file” name will be the name of the new .rda file.

#save to rda file
save(count, age, circumference, file = "mydata.rda")

Next we will remove our R environment variables using the command rm().

#remove variables
rm(age, circumference, count)

Now we can see that we no longer have saved variables in our R workspace.
no-saved-files

Now, we can check that our .rda file (myrda.rda) does in fact store our data by using the load() command.
Note: If we had not properly set our working directory, then we would have needed to provide a full path to the rda file. For example, “C:/Users/Documents/R files/myrda” rather than just “myrda”.

#load the rda file
load(file = "mydata.rda")

Great, now we can see that our variables are back in the R environment for use once more.

saved_files

Saving and loading data in R might be very useful when you’re working with large datasets that you want to clear from your memory, but you also would like to save for later. It also might be useful for long, complex R workflows and scripts. You can control the compression of the file using the settings ‘compress’ and ‘compression_level’.

That’s all for now!

Advertisements

Importing Data into R, part II

I recently downloaded the latest version of R Studio and noticed that their import dataset functionality had changed significantly. I had previously written about importing data HERE and wanted to provide an update for the current version of RStudio.

When you go to import data using R Studio, you get a menu like this.

rstudio-old-import

If you’re using the latest version of RStudio, when you click “From CSV” you’ll get a popup about downloading a new library ‘readr’.

readr

Once that has completed, you’ll see the new import data window (shown below).

new-import-screen

Okay, so first let’s make a simple comma delimited data file so we can test out the new import dataset process. I have made a simple file called “x-y-data.txt” as shown below. If you make this same file (no spaces, just a comma to separate the x column from the y column) then we can do this exercise together.

x-y-data

Now, let’s use the RStudio import to bring in the file “x-y-data.txt”. Here’s a screen grab of the import screen with my x-y dataset.

import-data

We can see that RStudio has used the first row as names, has recognized that it is a comma delimited file, and has read both x and y values as integers. Everything looks good, so I click “import”.

It was after this import process, that I had tried running some of my standard functions, such as making an empirical CDF (cumulative density function) and then I ran into problems. So let’s check the type of data we have imported.

# get the data structure
typeof(x_y_data)
#[1] "list"
class(x_y_data)
#[1] "tbl_df"     "tbl"        "data.frame"

While the old RStudio would have imported this as a matrix by default, this latest version of RStudio imports data as a data frame by default. Apparently RStudio has created their own version of a data frame called a “tbl_df” or tibble data frame. When you use the ‘readr’ package, your data is imported automatically as a “tbl_df”.

Now this isn’t necessarily a bad thing, in fact it seems like there is some nice functionality gained by using the “tbl_df” format. This change just broke some of my previously written code and it’s good to know what RStudio is doing by default.

If we wanted to get back to the matrix format, we can do this will a simple as.matrix function. From there we can verify it was converted using the typeof and class functions.

# convert to a matrix
data<-as.matrix(x_y_data)
#     x  y
#[1,] 1  2
#[2,] 2  4
#[3,] 3  6
#[4,] 4  8
#[5,] 5 10

typeof(data)
#[1] "integer"
class(data)
#[1] "matrix"

You can read more about the new Tibble structure at these websites:

https://blog.rstudio.org/2016/03/24/tibble-1-0-0/

http://www.sthda.com/english/wiki/tibble-data-format-in-r-best-and-modern-way-to-work-with-your-data

Enjoy!

Secret Santa Picker 2 using R

Last year I made a blog post about a Secret Santa picker HERE, but to use it required quite a bit of messing around with the code. So this year I decided to improve the whole thing by making it a function rather than a script. The function take two inputs, a list of names and a number of names. The code for the function is listed first followed by the code for the script used to call the function.

Here’s the function. Nothing needs to be changed in this code for it to run properly.

# make a function 
secret_santa <-function(npeople, names){
  
  # this 'flag' is used to determine if the
  # function stays in or out of the while function
  flag = "bad"
  
  # first list of names
  namelist1 = matrix(names, ncol = 1, nrow = npeople)
  fam = matrix(ncol = 1, nrow = npeople, NA)
  
  while (flag == "bad"){
    
    # names to choose from
    namelist2 = matrix(ncol = 1, nrow = npeople, NA)
    
    for (i in 1:npeople){
      #pick the first name
      if (i==1){
        xx2 = sample(names, (npeople-i+1), replace=FALSE)
      } else
        xx2 = sample(xx2, (npeople-i+1), replace=FALSE)
      
      if (i == npeople & xx2[1]==namelist1[i,1]){
        flag = "bad"
        
      }else if(xx2[1]!= namelist1[i,1]){
        namelist2[i,1] = xx2[1]
        flag = "good"
      } else{
        namelist2[i,1] = xx2[2]
        flag = "good"
        }
      
      
      #set up the new matrix with one less name
      used = which(xx2==namelist2[i])
      xx2[used] = "zzzzz"
      xx2 = sort(xx2)[1:(npeople-i)]
    }
    
    #flag
    #add "has" to the matrix
    has = matrix(ncol=1, nrow = npeople, "has")
    
    #build the final matrices
    final = cbind(namelist1, has, namelist2)	
    #the final results
    #final
    
    
  }
  final
}

Save this function as “secret-santa-function.R” and we’ll call it from our script. Okay, now let’s make our script.

# call the function from the script
source("secret-santa-function.R")

### Function input
### make a list of names
names = c("James","Nick","Emily","Natasha","Bob", "Teddy")
n = length(names)

#call the function
output <-secret_santa(n, names)
output 

The list of names is the only input needed. In the case above it’s ‘names = c(“James”,”Nick”,”Emily”,”Natasha”,”Bob”, “Teddy”)’. The other variable the function needs is the number of names, which is read automatically from the length function. That’s it, you’re done. Call the function from the script and you’ve got your names.

names

Make a bar plot with ggplot

The first time I made a bar plot (column plot) with ggplot (ggplot2), I found the process was a lot harder than I wanted it to be. This post steps through building a bar plot from start to finish.

First, let’s make some data. I’m going to make a vector of months, a vector of the number of chickens and a vector of the number of eggs. That’s random enough for this purpose.

# make some data
months <-rep(c("jan", "feb", "mar", "apr", "may", "jun", 
               "jul", "aug", "sep", "oct", "nov", "dec"), 2)
chickens <-c(1, 2, 3, 3, 3, 4, 5, 4, 3, 4, 2, 2)
eggs <-c(0, 8, 10, 13, 16, 20, 25, 20, 18, 16, 10, 8)
values <-c(chickens, eggs)
type <-c(rep("chickens", 12), rep("eggs", 12))
mydata <-data.frame(months, values)

If parts of the above code don’t make sense, take a look at my post on using the R functions seq (sequence), rep (repeat), and cbind (column bind) HERE.

Now let’s load the ggplot package.

library(ggplot2)

We want to make a plot with the months as the x-axis and the number of chickens and eggs as the height of the bar. To do this, we need to make sure we specify stat = “identity”. Here’s the basic code for this plot.

p <-ggplot(mydata, aes(months, values))
p +geom_bar()

Notice that you will get the error shown above, “stat_count() must not be used with a y aesthetic.” We forgot to specify that we want the height of the column to equal the value for that month. So let’s do it again.

p <-ggplot(mydata, aes(months, values))
p +geom_bar(stat = "identity")

barplot1

This time we get a plot, but it looks fairly ugly, and the months are out of order. In fact the months are in alphabetical order so let’s fix that first. If we investigate the months, we will see they have ordered levels.

mydata$months
#[1] jan feb mar apr may jun jul aug sep oct nov dec jan feb mar apr may
#[18] jun jul aug sep oct nov dec
#Levels: apr aug dec feb jan jul jun mar may nov oct sep

We can fix the order of this category by changing the factor. Here’s some code that will fix our problem.

mydata$months <-factor(mydata$months, 
                      levels = c("jan", "feb", "mar", "apr", "may", "jun",
                                 "jul", "aug", "sep", "oct", "nov", "dec"))

Now if we look at the levels again, we will see that they’re rearranged in the order that we want.

mydata$months
#[1] jan feb mar apr may jun jul aug sep oct nov dec jan feb mar apr may
#[18] jun jul aug sep oct nov dec
#Levels: jan feb mar apr may jun jul aug sep oct nov dec

Okay, let’s make our plot again, this time with the months in the correct order.

p <-ggplot(mydata, aes(months, values))
p +geom_bar(stat = "identity", aes(fill = type)) 

barplot2

Okay, now the months are working, but we realize we only have one set of columns being plotted. We should have two sets, ‘chickens’ and ‘eggs’. To fix this we need to specify some feature that separates them. We already created this in the “type” column when we made our data frame.

If we make the color of the graphs based off of the data category then we should get two sets of columns. In our data frame, we put our categories in the column named “type”. Fill is a property of bar plots. If we were making a line plot and we wanted to set the colors by the type of data we would use color = type rather than fill = type.

p <-ggplot(mydata, aes(months, values))
p +geom_bar(stat = "identity", aes(fill = type))

barplot3

Cool! Sort of. We have stacked bar plots, but I want them next to one another, not stacked. We can fix that with one more change to our code using dodge.

p <-ggplot(mydata, aes(months, values))
p +geom_bar(stat = "identity", aes(fill = type), position = "dodge")

barplot4

Finally, let’s spruce the plot up a little bit. We’ll adjust the x-axis label (xlab), y-axis label (ylab), title (ggtitle) and update the look using theme_bw().

p <-ggplot(mydata, aes(months, values))
p +geom_bar(stat = "identity", aes(fill = type), position = "dodge") +
  xlab("Months") + ylab("Count") +
  ggtitle("Chickens & Eggs") +
  theme_bw()

barplot5

The plot finally looks good and we’re done. Happy plotting!

Importing Data into R

One of the most important features we need to be able to do in R is import existing data, whether it be .txt files, .csv files, or even .xls (Excel files). If we can’t import data into R, then we can’t do anything. Okay let’s get started.

The spirit of this blog is that whatever I do here, should also work for someone working from home. Thus, we all need to work from the same text file, so we have to build a simple text file together to make this whole process work. Since this isn’t R code, I’ll just break it down into simple steps that are easy to follow.

Step 1: open notepad
Step 2: enter data as I have shown below (no spaces, use only commas)
rain
Step 3: save the file as ‘rain.txt’ on your Desktop

Okay, great now it’s time to get to work in R importing this data. We have two options, importing using the R Studio environment (the easy way), or importing using standard R functions.

The Easy way (Import through R Studio)

Step 1: Click the ‘Import Dataset’ button, then click ‘From Local File’
import_r
import_r2

Step 2: Navigate to the ‘rain.txt’ file located on your Desktop and click ‘open’. The next dialog box we get shows the values contained within our file, and different importing options. A few things to notice, ‘Name’ at the top has been set to “rain”, which will become the variable our data is stored as in R. The ‘Heading’ radio button has already been moved to ‘yes’ because R Studio has recognized our column headers (month, rain_mm, flow_cms). Additionally, the ‘Separator’ has been adjusted to ‘comma’ as we have made a comma delimited text file. All you have to do is just click ‘Import’.

import-dataset

Step 3: R Studio automatically opens the ‘rain’ dataset as a table in a new tab. R Studio also provides the snippet of code it used to import the data, which is great! You can copy that code and paste it into your R script file for future use.
rain_data
rstudio_code

That’s it! You’re a pro at importing data using R Studio.

 The Hard way (Import using R functions)

There’s lots of functions that can be used to import data into R: read.table, read.csv, read.csv2, read.delim, read.delim2 (among others). We’ll use read.table in this example.

To understand how this function works, let’s open up the R help by typing ?read.table.

# Get R help
?read.table

That should open up a help file through your web browser, or in the lower right ‘Help’ menu if you’re using R Studio. The main pieces of this function we need to set are ‘file’, ‘header’, and ‘sep’.
-The ‘file’ piece is the file name and file path we want to import.
-The ‘header’ piece is set to TRUE or FALSE based on whether or not there is a header within the file.
-The ‘sep’ piece describes the separator used within the file (in our case, a comma)

So, here’s how our code should look:

# Import the data
rain<-read.table("C:/Users/YOUR-NAME/Desktop/rain.txt", header = TRUE,
                   sep = ",")

Two things of note in the code above.
-The part of the path “YOUR-NAME” is based on your computer login settings. It might be something like ‘Tim’, ‘Jane’, ‘PeterC’, etc.
-I have defined this data as “rain” in R, using the rain <- bit of code. Here’s how the data looks in R.

# The rain data
#   month rain_mm flow_cms
#1      1     128    15000
#2      2      98    12000
#3      3      92    11000
#4      4      77     9800
#5      5      68     7600
#6      6      63     5800
#7      7      76     5500
#8      8      81     5700
#9      9      84     6200
#10    10     122     9500
#11    11     117    15000
#12    12     125     1700

That’s it! You’re a pro at importing data into R via the hard way! Happy coding!

 

2 y-axis plotting

A simple plotting feature we need to be able to do with R is make a 2 y-axis plot. First let’s grab some data using the built-in beaver1 and beaver2 datasets within R. Go ahead and take a look at the data by typing it into R as I have below.

# Get the beaver datasets
beaver1
beaver2

We’re going to plot the temperatures within both of these datasets, which we can see (after punching into R) is the third column.

First let’s check the length of these datasets and make sure they’re the same.

# Get the length of column 3
length(beaver1[,3])
length(beaver2[,3])

[1] 114
[2] 100

Since beaver1 is longer, we’ll only plot rows 1 through 100 of the temperature data, so that it is the same length as beaver2.

# Plot the data
plot(beaver1[1:100, 3], type ="l", ylab = "beaver1 temperature")

Cool, your plot should look like this.
beaver1

Now, let’s add that second dataset on the right y-axis. So, we have to have to create a plot on top of this plot using the command par(new = TRUE).

# Add the second y-axis
plot(beaver1[1:100, 3], type ="l", ylab = "beaver1 temperature")
par(new = TRUE)
plot(beaver2[,3], type = "l")

beaver1-2

Woah, this plot is ugly! We have 2 y-axis labels plotting, 2 y-axis values plotting, and 2 x-axis values and labels plotting. Let’s turn those off using the commands xaxt = “n” and yaxt = “n”.

# updated plot
plot(beaver1[1:100, 3], type ="l", ylab = "beaver1 temperature")
par(new = TRUE)
plot(beaver2[,3], type = "l", xaxt = "n", yaxt = "n",
     ylab = "", xlab = "")

beaver1-3

Okay, it’s still pretty ugly, so let’s clean it up. Let’s make the margins bigger on the right side of the plot, add a y2 axis label, add a title, change the color of the lines and adjust the x-axis label. Don’t forget the legend! Here’s the code:

# final plot
par(mar = c(5, 5, 3, 5))
plot(beaver1[1:100, 3], type ="l", ylab = "beaver1 temperature",
     main = "Beaver Temperature Plot", xlab = "Time",
     col = "blue")
par(new = TRUE)
plot(beaver2[,3], type = "l", xaxt = "n", yaxt = "n",
     ylab = "", xlab = "", col = "red", lty = 2)
axis(side = 4)
mtext("beaver2 temperature", side = 4, line = 3)
legend("topleft", c("beaver1", "beaver2"),
       col = c("blue", "red"), lty = c(1, 2))

beaver2

Woo! Looks good. That’s all for now.

ggplot2 (ggplot) Introduction

In this post I’ll briefly introduce how to use ggplot2 (ggplot), which by default makes nicer looking plots than the standard R plotting functions.

The first thing to know is that ggplot requires data frames work properly. It is an entirely different framework from the standard plotting functions in R. Let’s grab a default data frame in R called mtcars. Let’s confirm it’s a data frame using some code:

# Get the mtcars data types
class(mtcars)

R confirms that this is in fact a data frame.

# the output
[1] "data.frame"

Feel free to take a look at the data itself by just typing the name into R. For bevity, I won’t show the data in this post.

# Look at mtcars
mtcars

Next let’s define some standard plot function names in ggplot.
geom_point = scatterplot (points or solid lines)
geom_boxplot = boxplot
geom_bar = column plot
There’s many more (really cool) plot types, but I’ll stop here for now.

Let’s make our scatterplot. Here’s the code to make a standard plot. Don’t forget to load the package ggplot2 before running this code using the library function (install ggplot2 first if you haven’t done so before).

# Plot the data
library(ggplot2)
ggplot(mtcars, aes(hp, mpg)) + geom_point()

basic

Success! The code above seems strange at first, but let’s dive into how it works. First we call ggplot and provide the data frame name ‘mtcars’. Then we give the x & y variables using the aes command. Finally we specify we’re making a scatterplot by attaching + geom_point().

Now let’s make this look better! This is where the power of ggplot shines. It’s really easy to make a nice looking plot.

# Plot the data
p <- ggplot(mtcars, aes(hp, mpg))
p + geom_point() + labs (x = "Horsepower (hp)", y = "Miles per Gallon (mpg)") +
  ggtitle("My mtcars Plot")

basic2

We can see that the syntax is a bit different this time. We save the first ggplot call to a variable p (p for plot), but any variable will work. Then we attached more plotting features using p + ——. For this plot we added custom x and y axis labels and a title.

Next let’s make a change to the overall look of the plot, using what ggplot calls a theme. We’ll add theme_bw.

# Plot the data
p <- ggplot(mtcars, aes(hp, mpg))
p + geom_point() + labs (x = "Horsepower (hp)", y = "Miles per Gallon (mpg)") +
  ggtitle("My mtcars Plot") + theme_bw()

basic3

Finally, let’s spruce it up my coloring the points blue and making them bigger, while also making our axes and titles bigger. The code below makes this final plot.

# Make the final plot
p <- ggplot(mtcars, aes(hp, mpg))
p + geom_point(size = 3, color = "blue") + 
  labs (x = "Horsepower (hp)", y = "Miles per Gallon (mpg)") +
  ggtitle("My mtcars Plot") + theme_bw()+
  theme(axis.text = element_text(size = 12), 
        axis.title = element_text(size = 14),
        plot.title = element_text(size = 18, face = "bold"))

ggplot2 final

Hope this helped explain the basics of ggplot. Here’s the link to the ggplot2 documentation (click me).