一、基礎(chǔ)介紹

Storm是一個免費開源的分布式實時計算系統(tǒng)。分布式意味著Storm是一個集群，部署在多臺機(jī)器上。實時便是實時計算，相比于MapReduce的批處理，實時更關(guān)注于數(shù)據(jù)處理的速度和延時。

Apache Storm官網(wǎng)提供了各個版本的下載，體現(xiàn)為apache-storm-*.tar.gz，部署Storm時，直接將其解壓，并配置相關(guān)配置文件即可。注意到，Storm采用Clojure和Java語言編寫，Clojure也是運行在JVM之上的，所以環(huán)境上要保證安裝Java環(huán)境。

Storm運行時體現(xiàn)為Master-Worker集群。Master節(jié)點運行nimbus進(jìn)程，給Work節(jié)點分任務(wù)。Worker節(jié)點運行supervisor進(jìn)程，負(fù)責(zé)分配nimbus傳遞過來的任務(wù)，以啟動或停止worker進(jìn)程。nimbus和supervisor都是無狀態(tài)的，它們之間通過zookeeper來協(xié)調(diào)任務(wù)，也就是將狀態(tài)信息存放在zookeeper中。

Storm的集群部署為：

二、基礎(chǔ)環(huán)境

# Linux操作系統(tǒng)版本
root@linux:# lsb_release -a 
No LSB modules are available.
Distributor ID:  Ubuntu
Description:  Ubuntu 18.04.2 LTS
Release:  18.04
Codename:  bionic
# python版本
root@linux:# python --version
Python 2.7.17
root@linux:# python3 --version
Python 3.6.9
# java版本
root@linux:# java -version
openjdk version "1.8.0_272"
OpenJDK Runtime Environment (build 1.8.0_272-8u272-b10-0ubuntu1~18.04-b10)
OpenJDK 64-Bit Server VM (build 25.272-b10, mixed mode)

三、Zookeeper安裝

1. 下載Zookeeper包，解壓并部署在/opt目錄下

tar -xvf apache-zookeeper-3.7.1-bin.tar.gz 
mkdir /opt/zookeeper
chmod 777 /opt/zookeeper/
mv apache-zookeeper-3.7.1-bin.tar.gz /opt/zookeeper/

2. 配置zoo.cfg文件

# The number of milliseconds of each tick
# 心跳時間，單位毫秒
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
# Leader和Follower初始連接時最大的心跳數(shù)
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
# 保存Zookeeper數(shù)據(jù)的目錄
dataDir=/opt/zookeeper/zkdata
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60

3.啟動 Zookeeper 服務(wù)端

root@linux:/opt/zookeeper/apache-zookeeper-3.7.1-bin/bin# ./zkServer.sh start
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/apache-zookeeper-3.7.1-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@linux:/opt/zookeeper/apache-zookeeper-3.7.1-bin/bin#

4. 查看進(jìn)程

root@linux:/opt/zookeeper/apache-zookeeper-3.7.1-bin/bin# jps
18706 Jps
18670 QuorumPeerMain #Zookeeper服務(wù)進(jìn)程
root@linux:/opt/zookeeper/apache-zookeeper-3.7.1-bin/bin# 
查看狀態(tài)
root@linux:/opt/zookeeper/apache-zookeeper-3.7.1-bin/bin# ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/apache-zookeeper-3.7.1-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: standalone

四、Storm安裝

1. 修改conf/storm.yaml文件，修改為本機(jī)的IP地址

########### These MUST be filled in for a storm configuration
 storm.zookeeper.servers: #Zookeeper主機(jī)列表
     - "30.0.0.218"
 nimbus.seeds: ["30.0.0.218"] #master候選者

2. 打開/etc/profile文件，增加如下：

export PATH=$PATH:/opt/apache-storm-2.3.0/bin

3. 執(zhí)行命令：source /etc/profile。
4. 按照順序啟動：

storm nimbus &
storm supervisor &
storm ui &

5. 查看啟動進(jìn)程：

root@linux:# jps
22817 UIServer
22549 Nimbus
22709 Supervisor
20775 QuorumPeerMain
23039 Jps

可能會遇到的問題：

端口沖突問題：
root@linux:/opt# Running: java -server -Ddaemon.name=ui -Dstorm.options= -Dstorm.home=/opt/apache-storm-2.3.0 -Dstorm.log.dir=/opt/apache-storm-2.3.0/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file= -cp /opt/apache-storm-2.3.0/*:/opt/apache-storm-2.3.0/lib/*:/opt/apache-storm-2.3.0/extlib/*:/opt/apache-storm-2.3.0/extlib-daemon/*:/opt/apache-storm-2.3.0/lib-webapp/*:/opt/apache-storm-2.3.0/conf -Xmx768m -Djava.deserialization.disabled=true -Dlogfile.name=ui.log -Dlog4j.configurationFile=/opt/apache-storm-2.3.0/log4j2/cluster.xml org.apache.storm.daemon.ui.UIServer
Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8080
  at org.apache.storm.daemon.ui.UIServer.main(UIServer.java:183)
Caused by: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8080
  at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
  at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
  at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
  at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
  at org.eclipse.jetty.server.Server.doStart(Server.java:394)
  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
  at org.apache.storm.daemon.ui.UIServer.main(UIServer.java:179)
Caused by: java.net.BindException: Address already in use
  at sun.nio.ch.Net.bind0(Native Method)
  at sun.nio.ch.Net.bind(Net.java:461)
  at sun.nio.ch.Net.bind(Net.java:453)
  at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
  at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
  at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
  ... 7 more


執(zhí)行命令：lsof -i:8080，可以看到8080被zookeeper占用。
修改zookeeper的zoo.cfg文件，添加如下：
admin.serverPort=8008
端口沖突解決。