文書の過去の版を表示しています。
Elasticsearch クラスタ設定
Requirements
- You must first follow the installation and configuration steps on each node.
- The minimum size of an Elasticsearch cluster is 3 nodes and it must always grow in odd numbers in order to make use of the quorum system and guarantee data integrity.
- Ensure that you have connectivity between all 3 nodes and that ports 9200 and 9300 are accessible between each and every node.
Remember to configure the firewall of each node to allow connection through these port numbers.
Configuration
Stop the Elasticsearch service on each and every node:
systemctl stop elasticsearch.service
Modify the following lines in the configuration file /etc/elasticsearch/elasticsearch.yml :
#discovery.seed_hosts: ["host1", "host2"] #cluster.initial_master_nodes: ["host1", "host2"]
Uncomment the lines and add the IP addresses or URLs of each node:
discovery.seed_hosts: ["host1", "host2", "host3"] cluster.initial_master_nodes: ["host1", "host2", "host3"]
Example with IP addresses:
discovery.seed_hosts: ["172.42.42.101", "172.42.42.102", "172.42.42.103"] cluster.initial_master_nodes: ["172.42.42.101", "172.42.42.102", "172.42.42.103"]
Make sure that the line cluster.initial_master_nodes is defined only once in the configuration file, in some cases the same line appears in two different blocks of the same file.
Before starting the service, because the nodes were started for the first time on their own (standalone), the contents of the data folder (by default /var/lib/elasticsearch/) must be deleted in order to start the cluster for the first time. Do this with the command:
rm -rf /var/lib/elasticsearch/*
Now it is time to start the services on each and every node. Start and check that they are running with the commands:
systemctl start elasticsearch.service && systemctl status elasticsearch.service
You should get an output similar to:
Once the services have been started, you must confirm that the 3 nodes are joined to the cluster correctly, so when executing the following command on any of the nodes, the same response should be given:
curl -XGET http://127.0.0.1:9200/_cat/nodes
Check again the firewall configuration always taking into account that the nodes should communicate through ports 9200 and 9300 and that from the PFMS server and the PFMS Web Console should be able to access port 9200 as well. With these steps you will have already installed and configured the Elasticsearch cluster to be used as Pandora FMS log storage engine.
Data models and templates
Before putting into production an environment, either a single node or a data cluster, it is recommended to apply the corresponding configurations to this node or cluster according to its use. In the case of the indexes generated by Pandora FMS, the most effective way to do it is defining a template to define the configuration of the fields and the stored data.
Templates are configurations that are only applied at the time of index creation. Changing a template will have no impact on existing indexes.
- To create a basic template follow the instructions in “Data models and templates for a node”.
- To define a multinode template you must take into account the following information:
- When configuring the template (JSON format), you need to configure as many searches as you have nodes, however to correctly configure the replicas you must subtract 1 from the number of nodes in the environment.
For example, in a Pandora FMS environment with Elasticsearch with 3 configured nodes, when you modify the number_of_search and number_of_replicas fields it should look like this:
{
"index_patterns": ["pandorafms*"],
"settings": {
"number_of_shards": 3,
"auto_expand_replicas" : "0-1",
"number_of_replicas" : "2"
},
This is a very basic definition, in order to correctly define the sizing of the Elasticsearch environment it is advisable to take into account the factors described in this article:
From the command line you can list the templates of the environment by executing:
curl -X GET "localhost:9200/_cat/templates/*?v=true&s=name&pretty"
You can also view the details of a template, for example the one we have created for pandorafms by running it:
curl -X GET "localhost:9200/_template/pandorafms*?pretty"
which will return in JSON format the configuration you have defined.
You can perform these operations through the Elasticsearch interface in Pandora FMS using the native Elasticsearch commands.
- PUT _template/<template_name> {json_data}: allows you to enter the data of the template to be created.
- GET _template/><template_name>: allows you to display the created template.
