Wednesday, April 01, 2015

Running the new IBM Open Platform with Apache Hadoop in Docker Container

IBM has just announced it's new Open Platform with Apache Hadoop.
IBM® Open Platform with Apache Hadoop builds the platform for big data projects and provides the most current Apache Hadoop open source content. IBM offers this 100% open source Apache distribution as a free download as well as a supported offering for all your Hadoop workloads.
IBM Open Platform with Apache Hadoop
  • Native support for rolling upgrades for Hadoop services
  • Support for long-running applications within YARN for enhanced reliability & security
  • Heterogeneous storage in HDFS for in-memory, SSD in addition to HDD
  • Apache Spark in-memory distributed compute engine for dramatic performance increases over MapReduce and simplifies developer experience, leveraging Java, Python & Scala languages
  • Ambari operational framework for provisioning, managing & monitoring Apache Hadoop clusters
  • Apache Hadoop projects included: HDFS, YARN, MapReduce, Ambari, Hbase, Hive, Oozie, Parquet, Parquet Format, Pig, Snappy, Solr, Spark, Sqoop, Zookeeper, Open JDK, Knox, Slider
If you are interested in playing with it and familiarize with the new product and it's components, I have built an IOP Single Node Docker container that you could use and deployed it to Docker Hub.

Downloading the docker image:
docker pull lresende/iop:4.0

Running the image:
The command below run the IOP image, and do the mapping to the exposed ports used by Ambari and it's components.
sudo docker run -d --privileged=true -P -p 10000:10000 -p 11000:11000 -p 2122:2122 -p 50010:50010 -p 50020:50020 -p 50030:50030 -p 50070:50070 -p 50075:50075 -p 50090:50090 -p 50111:50111 -p 60010:60010 -p 8000:8000 -p 8020:8020 -p 8021:8021 -p 8081:8081 -p 8088:8088 -p 8440:8440 -p 8043:8043 -p 8441:8441 -p 8670:8670 -p 8888:8888 --name iop -h ambari.luck.com lresende/iop:4.0 /etc/bootstrap.sh -d


Once IOP docker image is up and running, access the Ambari console and start all services (click Actions->Start All in the left, bottom side of the screen).
http://<host>:8081

And, if you want to do it yourself, all the source code used to build this IOP Single Node Cluster is also available in the github repository: docker-ambari-iop.

2 comments:

  1. Works perfectly, thanks!

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete