【ELK】ELK安装部署文档


本博客文章如无特别说明,均为原创!转载请注明出处:Big data enthusiast(http://www.lubinsu.com/)

本文链接地址:【ELK】ELK安装部署文档(http://www.lubinsu.com/index.php/archives/395)

● 摘要:
ELK是当前很流行的日志分析系统,Elasticsearch是一款基于Apache Lucene的开源分布式引擎。Logstash是用于日志的收集、转换、并输出到ES,其中有丰富的插件用于集成诸如Filebeat、Flume、Kafka、Log4J等各种外部数据源,还能输出到各种目标存储器中。Kibana是基于ES的分析与可视化平台,我们可以通过Kibana在ES中搜索、查看各类索引并制作出各种图表。另外如果需要增加安全性我们可以通过安装X-Pack来实现。

● 工作原理图

● 安装Elasticsearch 8台虚拟机:
版本:Elasticsearch 5.5.0、jdk1.8.0_11
插件:X-Pack
安装步骤参考链接:https://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html

1. curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.0.tar.gz
2. tar -xvf elasticsearch-5.5.0.tar.gz
3. vim confit/elasticsearch.yml
cluster.name: my-application
node.name: hostname
node.master: true
node.data: true
path.data: /opt/esuser/tools/data
path.logs: /opt/esuser/tools/logs
discovery.zen.ping.unicast.hosts: [“master”,”slaver01″, “slaver02”]

4. cd elasticsearch-5.5.0/bin
5. ./elasticsearch

● 安装Kibana 1台
版本:Kibana 5.5.0、jdk1.8.0_11
安装步骤参考链接:https://www.elastic.co/guide/en/kibana/current/targz.html
1. wget https://artifacts.elastic.co/downloads/kibana/kibana-5.5.0-linux-x86_64.tar.gz

2. tar -xzf kibana-5.5.0-linux-x86_64.tar.gz
3. cd kibana/
4. vim kibana/config/kibana.yml
添加:elasticsearch.url: “http://192.168.0.181:9200”
5. ./bin/kibana

● 安装X-Pack
版本:X-Pack 5.5.0、jdk1.8.0_11
安装步骤参考链接:https://www.elastic.co/downloads/x-pack
网络不好时,可先下载后安装,参考链接:https://www.elastic.co/guide/en/x-pack/current/installing-xpack.html
1. Install X-Pack into Elasticsearch
bin/elasticsearch-plugin install x-pack
2. Start Elasticsearch
bin/elasticsearch
3. Install X-Pack into Kibana
bin/kibana-plugin install x-pack
4. Start Kibana
bin/kibana
5. Navigate to Kibana at http://localhost:5601/
6. Log in as the built-in elastic user with the password changeme.

● 生成传输加密证书
1. filebeat
openssl req -subj ‘/CN=192.168.0.181/’ -x509 -days $((100*365)) -batch -nodes -newkey rsa:2048 -keyout /opt/esuser/tools/logstash/key/filebeat-181.key -out /opt/esuser/tools/logstash/certs/filebeat-181.crt
2. logstash
openssl req -subj ‘/CN=192.168.0.181/’ -x509 -days $((100*365)) -batch -nodes -newkey rsa:2048 -keyout /opt/esuser/tools/logstash/key/logstash.key -out /opt/esuser/tools/logstash/certs/logstash.crt

如果filebeat报如下错误:
2017-07-18T14:45:37+08:00 ERR Connecting error publishing events (retrying): x509: cannot validate certificate for 192.168.0.181 because it doesn’t contain any IP SANs
则在文件中添加对应的IP,重新生成证书即可:vim /etc/pki/tls/openssl.cnf
[ v3_ca ]
subjectAltName = IP:192.168.0.181

● 安装Logstash 3台虚拟机
版本:Logstash 5.5.0、jdk1.8.0_11
安装步骤参考链接:https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

1. wget https://artifacts.elastic.co/downloads/logstash/logstash-5.5.0.tar.gz
2. tar -xvf logstash-5.5.0.tar.gz
3. 配置处理filebeat过来的数据 vim logstash/config/filebeat.conf:【Logstash】接收filebeat日志配置
4. 启动:bin/logstash -f config/filebeat.conf

发表评论

电子邮件地址不会被公开。 必填项已用*标注