banner
LegalGeek

LegalGeek

搞产品的法律人
HomeArchives

Installing ELK on M1 Chip MacBook Pro

#macOS#ElasticSearch194
AI Translation
This post is translated from Chinese into English through AI.View Original
AI-generated summary
This article provides instructions on how to install and optimize ELK (Elasticsearch, Kibana, Logstash) on a MacBook Pro with an M1 chip. It includes steps for installing Elasticsearch, Kibana, and Logstash using brew, as well as optimizing memory usage and setting up access passwords. The article also provides instructions for configuring Kibana to display in Chinese, limiting memory usage for Elasticsearch, and setting up password protection for Elasticsearch and Kibana. Finally, it includes instructions for installing Logstash using brew.

Abstract: Install elasticsearch, kibana, and logstash suitable for M1 chip, optimize memory usage, and set access password.

Install Elasticsearch#

Use brew to install Elasticsearch suitable for M1 chip.

# brew tap elastic/tap

brew install elastic/tap/elasticsearch-full

After successful installation, usage help is displayed:

# Installation log
==> Installing elasticsearch-full from elastic/tap
Warning: Tried to install empty array to /opt/homebrew/etc/elasticsearch/jvm.options.d
==> codesign -f -s - /opt/homebrew/Cellar/elasticsearch-full/7.17.1/libexec/modules/x-pack
==> find /opt/homebrew/Cellar/elasticsearch-full/7.17.1/libexec/jdk.app/Contents/Home/bin
==> Caveats

# Related installation file locations
Data:    /opt/homebrew/var/lib/elasticsearch/elasticsearch_zhangminglei/
Logs:    /opt/homebrew/var/log/elasticsearch/elasticsearch_zhangminglei.log
Plugins: /opt/homebrew/var/elasticsearch/plugins/
Config:  /opt/homebrew/etc/elasticsearch/

# Run in the background and start automatically on boot
  brew services start elastic/tap/elasticsearch-full

# Run in the current command line, service stops when terminal is closed
  elasticsearch

Start the background service:

brew services start elastic/tap/elasticsearch-full

After running, verify by accessing: http://localhost:9200/, the following response content appears:

// 20220315232052
// http://localhost:9200/

{
  "name": "LegalgesMacBook.lan",
  "cluster_name": "elasticsearch_zhangminglei",
  "cluster_uuid": "fB4xhSO1TrO0yh1I4h7qaw",
  "version": {
    "number": "7.17.1",
    "build_flavor": "default",
    "build_type": "tar",
    "build_hash": "e5acb99f822233d62d6444ce45a4543dc1c8059a",
    "build_date": "2022-02-23T22:20:54.153567231Z",
    "build_snapshot": false,
    "lucene_version": "8.11.1",
    "minimum_wire_compatibility_version": "6.8.0",
    "minimum_index_compatibility_version": "6.0.0-beta1"
  },
  "tagline": "You Know, for Search"
}

Install Kibana#

Use brew to install Kibana suitable for M1 chip.

# brew tap elastic/tap

brew install elastic/tap/kibana-full

After successful installation, it prompts:

# Installation process log
==> Installing kibana-full from elastic/tap
==> Caveats
Config: /opt/homebrew/etc/kibana/
If you wish to preserve your plugins upon upgrade, make a copy of
/opt/homebrew/opt/kibana-full/plugins before upgrading, and copy it into the
new keg location after upgrading.

# Run in the background and start automatically on boot
To start elastic/tap/kibana-full now and restart at login:
  brew services start elastic/tap/kibana-full

# Run in the current command line, service stops when terminal is closed
Or, if you don't want/need a background service you can just run:
  kibana

# Installation result summary
==> Summary
🍺  /opt/homebrew/Cellar/kibana-full/7.17.1: 51,438 files, 720MB, built in 30 seconds
==> Running `brew cleanup kibana-full`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

Start the background service:

brew services start elastic/tap/kibana-full

Access http://localhost:5601/ in the browser to view the running interface, and change the system interface to Chinese:

# Switch to the configuration file directory
cd /opt/homebrew/etc/kibana/

# Edit the configuration file
nano kibana.yml

# Change the last line to:
i18n.locale: "zh-CN"

Restart the brew service:

brew services restart elastic/tap/kibana-full

Adjust System Memory Usage#

The ES service by default occupies more than half of the system's memory, which can be limited by modifying the configuration file:

The file to be modified is in the service configuration path: /opt/homebrew/etc/elasticsearch/

The file to be modified: jvm.options

Before modification:

## jvm.options.d directory containing these lines:
##
## -Xms4g
## -Xmx4g
##

After modification:

## jvm.options.d directory containing these lines:
##
-Xms4g
-Xmx4g
##

Restart the ES service, and you can see that the Java memory usage has been limited in the system's Activity Monitor:

brew services restart elastic/tap/elasticsearch-full

Set Password Protection#

Note: You can refer to the official documentation: https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html

  • step1: Shut down the running services of Elasticsearch and Kibana.

  • step2: Modify the elasticsearch.yml file to enable password protection and set it to single-node.

# Enter the configuration folder
/opt/homebrew/etc/elasticsearch/

# Edit the configuration file
elasticsearch.yml

# Append the following 2 lines at the end of the file:
xpack.security.enabled: true
discovery.type: single-node
  • step3: Use the built-in tool of Elasticsearch to automatically generate account and password.
#1 Start the Elasticsearch service:
brew services start elastic/tap/elasticsearch-full

#2 Enter the ES executable file directory:
 /opt/homebrew/Cellar/elasticsearch-full/7.17.1/bin

#3 Open the terminal in the above directory and execute the command to automatically generate the password:
./elasticsearch-setup-passwords auto

#4 Record the account and password output in the console, which will be used later.
Example of generated account and password:
Changed password for user kibana_system
PASSWORD kibana_system = n9zXw2UnJwvWdrJ4ZPcg

Changed password for user elastic
PASSWORD elastic = Lhder7yQVB8z3rzdFoKY
  • step4: Configure the automatically generated account and password in Kibana to allow it to access ES.
#1 Enter the Kibana configuration file directory:
 /opt/homebrew/etc/kibana/

#2 Modify the Kibana configuration file, append content:
elasticsearch.username: "kibana_system"

#3 Enter the Kibana executable file directory:
 /opt/homebrew/Cellar/kibana-full/7.17.1/bin

#4 Open the terminal in the above directory and execute in sequence:
    ./kibana-keystore create
    ./kibana-keystore add elasticsearch.password
    Enter the above generated kibana_system password.

#5 Restart the Kibana service:
brew services start elastic/tap/kibana-full
  • step5: Use the automatically generated elastic account and password to log in to Kibana.
Address: http://localhost:5601/
Account: elastic
Password: Lhder8yQVB8z3rzdFoKY

*The password can be changed in Kibana.

Install Logstash#

Use brew to install Logstash suitable for M1 chip:

# brew tap elastic/tap

brew install elastic/tap/logstash-full

Installation success log:

Please read the getting started guide located at:
  https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html

To start elastic/tap/logstash-full now and restart at login:
  brew services start elastic/tap/logstash-full
Or, if you don't want/need a background service you can just run:
  logstash
==> Summary
🍺  /opt/homebrew/Cellar/logstash-full/7.17.1: 14,003 files, 604.8MB, built in 13 seconds
==> Running `brew cleanup logstash-full`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.

Configure username and password in logstash.yml:

Path: /opt/homebrew/etc/logstash

xpack.monitoring.enabled: true

xpack.monitoring.elasticsearch.username: logstash_system

xpack.monitoring.elasticsearch.password: *****

xpack.monitoring.elasticsearch.hosts: ["http://127.0.0.1:9200"]

Start the service:

brew services start elastic/tap/logstash-full

Verify installation success:

# Enter the logstash directory
cd /opt/homebrew/Cellar/logstash-full/7.17.1/bin

# Execute the following command, wait for the input cursor to appear
bin/logstash -e 'input { stdin { } } output { stdout { } }'

# After entering hello, the system's feedback:
{
      "@version" => "1",
       "message" => "hello",
    "@timestamp" => 2022-05-29T06:28:37.021Z,
          "host" => "izwz9cte3psk7ixd8kospzz"
}

Create and use a pipeline configuration file:
A Logstash management typically has one or more input, filter, and output plugins, for example:

//hello.conf

input { stdin { } }
output {
    elasticsearch {
     hosts => "http://127.0.0.1:9200"
     index => "logs"
     user => "elastic"
     password => "************"
   }
  stdout {}
  }

Use the pipeline:

# Execute
logstash -f hello.conf

# Input:
hello world

# Feedback:
{
          "host" => "Legalgeeks-MacBook.local",
    "@timestamp" => 2022-03-20T05:50:48.988Z,
      "@version" => "1",
       "message" => "hello world"
}
Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.