One of my models is horribly slow and was going to take about two weeks to run. Although two week isn’t really that bad, I don’t like waiting… So I decided to try NeCTAR. Running my models on a Virtual Machine sounded like an excellent solution. Just one little issue: I had absolutely no idea where to get started.
If you are like me (not familiar with virtual machines, cloud computing and command line interfaces), trying to use NeCTAR can be quite offputting. Especially if you don’t want to spend three days figuring out how it works. But lucky you, I did.
Running Rstudio in the cloud for total beginners
1. Logging in to your Dashboard.
- Go here: https://dashboard.rc.nectar.org.au
- Log in with your AAF credentials (your uni login). If you don’t have these, it means you will never ever ever be able to get in. Stop reading this and go to Amazon Web Services, they will give you a year of cloud computing services for free.
- Once logged in, you will see your dashboard. Just click around for a bit to see where everything is, you don’t have to know what any of these things mean yet. Note: make sure you are always working in the right project. If this is your first time in the NeCTAR Dashboard, don’t worry about this. You should only have one project, which is a trial project that runs for about 3 months.
Your NeCTAR Dashboard will look similar to this - click on picture to enlarge
2. Creating a Key Pair
A key pair is an added safety feature you will need to start an instance (an instance is where your virtual machine will be running) and get access to your instance. A key pair has two parts: a public part and a private part. The public part is like the lock on your door: public to everyone, but only the person with the key (the private part), can get access to what’s behind it. Just like with physical keys, don’t lose your private key.
- On your Dashboard, go to: Project > Compute > Access & Security. At the top left (sorf of) of the screen, choose Key Pairs. Your screen should now show something like this:
- Click on Create Key Pair > choose a name > Create Key Pair. A window will pop up and prompt you to download your <yourkeypairname>.pem. Download it and don’t lose it. This is your private key.
- That’s all. The public key will be stored somewhere on NeCTAR, so you don’t have to worry about that.
Security groups are the security guards of your virtual machine: they decide who gets in and who doesn’t, and how. We are going to make our own security group. Note: NeCTAR is very careful with security, so please read this as well: https://support.rc.nectar.org.au/docs/security-guidelines.
- Go to: Project > Compute > Access & Security. At the top of the screen, choose Key Pairs. Your screen should now show something like this:
- Click on Create Security Group. Choose a name and description and click on Create Security Group. I doesn’t matter what you fill in at this point, you can always go back and change or delete it.
- Your new group should now be added to the list. Next, we are going to add some rules. These rules will explain to our virtual security guards who get’s in (which IP’s) and how (through which ports).
- Click on Manage Rules > Add rule (in the next screen at top right).
- Fill out all the fields as in the picture above and click Add. Port 22 stands for SSH access (instead of ‘Custom TCP Rule’ you can also choose ‘SSH’ in the top field). CIDR states which computer (IP) addresses get access. 0.0.0.0/0 means that everyone can get in.
- Add two more rules. Keep all the fields the same as in the picture above, but for the second rule fill out Port 80, which stands for HTTP. For the third rule, use Port 8787, the default access port of the Rstudio server.
- You have now told the Security Group that it should only allow access to the virtual machine through port 22, 80 and 8787, but every IP address can get in.
- Your virtual machine will be running on what is called an instance. To launch an instance go to Project > Compute > Instances. At the top right of the screen, choose Launch Instance. This window will come up:
- In the Details tab, choose a name for you instance. Also, you will have to choose a flavour. I recommend to choose m1.small for now, as that will take up half the allocated space in your trial project. Under image name, choose NeCTAR Ubuntu Trusty (or just do what it shows in the screenshot above).
- Under Access & Security, choose the Key Pair you made in step 2 and select the Security Group you made in step 3.
- Click Launch. Now your virtual machine will start loading. Once the Power State says Running, you can move on to the next step.
Now this step is a little bit trickier, because it depends on what computer you are currently using. Here, I’m going to explain how to do this using a Windows computer. If you are running a Mac (or Linux or something else Unixy), it’s actually more straigthforward, just open a terminal and type this:
ssh -i mykeypairname ubuntu@XX.XX.XX.XX
(you can find you IP address on the Instances page) and wait for the Windows users to catch up in step 6.
- Windows users: Download PuTTY and PuTTYgen here. PuTTY is an SSH client for Windows, you will need this to get access to your instance. You are going to use PuTTYgen to translate the <yourkeypairname>.pem file into something Windows understands. Silly Windows.
- Open PuTTYgen.
- Click Load, and load the <yourkeypairname>.pem file you downloaded earlier. Now click Save private key. Give the new file the same name as your old .pem file (but the extension should be .ppk on your new file).
- Close PuTTYgen and open PuTTY. We are now ready to SSH our way into the instance. PuTTY should open up in the Session screen.
- Under Host Name, fill out: ubuntu@XX.XX.XX.XX. You can find you instance’s IP address on the Instances page and place that instead of the XX.XX.XX.XX.
- Make sure port 22 is selected and that connection type SSH is selected. Don’t worry about the rest.
- Next go to Connection > SSH > Auth:
- In the Authorization screen, choose Browse. Open the <yourkeypairname>.ppk file we created earlier using PuTTYgen. Once you have done that, click Open in the PuTTY screen. We are now SSHing our way into the instance. Yay!
- If a warning pops up, click Yes (assuming you trust the connection we are making). It’s just doing that because it has never made this connection before.
Mac users: you can join again!
- We have now connected to our instance. Your screen should look something like this:
- If it is asking for a username or something like that, type ‘ubuntu’.
- We are now in our virtual machine with ubuntu as an operating system. We can start telling our operating system things we want it to do, such as running R.
- After some websearching I came accross this awesome bit of code provided by yhat. Type the following bits of code in the ubuntu server and it will install R and Rstudio:
# adding a username. follow the prompts (no need to fill out your name and address) sudo adduser USERNAME # update sudo apt-get update # install r sudo apt-get install r-base # install rstudio sudo apt-get install gdebi-core sudo apt-get install libapparmor1 # check rstudio.com for the newest version wget http://download2.rstudio.org/rstudio-server-0.98.1103-amd64.deb sudo gdebi rstudio-server-0.98.1103-amd64.deb # make sure everything is working sudo rstudio-server verify-installation
- Now open your browser and go to: XX.XX.XX.XX:8787. It will prompt you for the username and password you just created with ‘sudo adduser’. Fill those out and you’re in!