Comments ( 2 )

1. Amazon Web Services | wiki.davelevy.infoPosted on 9:06 am 22nd November 2013

   […] redland rdf using Ubuntu’s package manager which were also required. I also experimented with Hadoop and VNC. In November 2013, I needed a WordPress […]
2. DavePosted on 1:55 pm 17th July 2017

   I copied this from the AWS article as I propose to extend that article and increase it's focus on AWS. It was originally sub titled "Experimenting with Hadoop" Single Node Cluster I plan to upload a single node hadoop cluster to AWS. I plan to locate it on EBS so it can survive a system halt. I originally planned to create it on a Virtual Box image, and upload the image. I concluded that the process of creating an image from scratch when the distro authors have AMIs available is too high. One would need a good reason.  20th Sept 2011 Two Node Cluster Michael Noll has written a tutorial for multi-node clusters, so I plan to make one of these. The first thing is that the two nodes need to be able to talk to each other, this requires enabling in the firewall rules. It is not possible to change the ruleset name for an instance once created. It is possible to change the rules and so I have created a private AMI with a single node hadoop server configured with firewalls that permit ping, http and ssh. I later enhanced these rules to include ports 54310-54311 and other ports mentioned on the Cloudera Hadoop Ports Quick Reference page. If I want to monitor my AMIs with Zabbix, I will need to add the Zabbix ports to the firewall rules. Any way I have distributed the master public key to slave node. I now need to consider how dynamic to make the name discovery. I could use dyndns. Probably best to just get it done.