'분류 전체보기'에 해당되는 글 93건

use mydb;

alter EVENT expireRow

ON SCHEDULE

EVERY 1 HOUR

DO 

DELETE FROM thatTable 

WHERE TIMESTAMPDIFF(DAY, date , NOW()) > 7

==

SHOW EVENTS FROM mydb;

sudo sh -c "ulimit -n 64000 && exec su $LOGNAME"

exit

server {

  listen 80 default_server;

  listen [::]:80 default_server;

  server_name _;

  # Pass requests for / to localhost:8080:

   location / {

          proxy_set_header X-Real-IP $remote_addr;

          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

          proxy_set_header X-NginX-Proxy true;

          proxy_pass http://127.0.0.1:8080/;

          proxy_set_header Host $http_host;

          proxy_cache_bypass $http_upgrade;

          proxy_redirect off;

   }

   error_page 404 /404.html;

          location = /40x.html {

   }

   error_page 500 502 503 504 /50x.html;

          location = /50x.html {

   }

}

Install

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/

Setup

sudo yum install -y mongodb-org-shell-3.6.1 mongodb-org-mongos-3.6.1

sudo mkdir /var/log/mongodb

sudo chown -R centos:centos /var/log/mongodb/

mongos --configdb configRS/ip-10-20-0-71.ec2.internal:27019,ip-10-20-0-57.ec2.internal:27019,ip-10-20-10-155.ec2.internal:27019 --logpath /var/log/mongodb/mongos.log --fork

https://www.elastic.co/guide/en/elasticsearch/reference/current/enabled.html

0. Delete your index pattern

1. Get default template

If you are using logstash, then you can find default template from kibana Dev API

GET _template/logstash

2. Add index disable setting

If you don't want to use "foo" field for index, you can tell ES "do not indexing this field"

copy default template and add "foo": {"enabled": false} this setting on "properties" of default template

"properties": {

  "foo": {"enabled": false}

}

3. put new settings

PUT _template/logstash

{

  ... changed whole new template ...

}

* If you want to remove .keyword things, setting "string_fields" under mappings._default_.dynamic_templates

(WARN: keyward is for keyword search of target field. if you want keyword search for specific field, you can setting it manually in template.json)

"string_fields": {
  "match": "*",
  "match_mapping_type": "string",
  "mapping": {
    "type": "text",
    "norms": false
  }
}

* Other way to using template.json file instead of changing _template/logstash

1. make template.json from GET _template/logstash

2. add template.json to logstash.conf

3. restart logstash

elasticsearch {
  hosts => ["http://domain:80"]
  template => "/etc/logstash/template.json"
}

template.json full version example)

{
  "order": 0,
  "version": 60001,
  "index_patterns": [
    "logstash-*"
  ],
  "settings": {
    "index": {
      "refresh_interval": "5s"
    }
  },
  "mappings": {
    "_default_": {
      "dynamic_templates": [
        {
          "message_field": {
            "path_match": "message",
            "match_mapping_type": "string",
            "mapping": {
              "type": "text",
              "norms": false
            }
          }
        },
        {
          "string_fields": {
            "match": "*",
            "match_mapping_type": "string",
            "mapping": {
              "type": "text",
              "norms": false
            }
          }
        }
      ],
      "properties": {
        "@version": {
          "enabled": false
        },
        "offset": {
          "enabled": false
        },
        "tags": {
          "enabled": false
        }
      }
    }
  },
  "aliases": {}
}

Errors

2017-12-19T16:27:12+09:00 ERR  Failed to publish events caused by: read tcp [::1]:48818->[::1]:5044: i/o timeout

2017-12-19T16:27:12+09:00 ERR  Failed to publish events caused by: client is not connected

2017-12-19T16:27:13+09:00 ERR  Failed to publish events: client is not connected

In my case,

It is caused from elasticsearch error.

This is logstash log. (see logstash.yml. "log level", and change it to info)

[2017-12-19T16:53:22,168][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"index_create_block_exception", "reason"=>"blocked by: [FORBIDDEN/10/cluster create-index blocked (api)];"})

My AWS ES status was yellow(you should have minimum two ES instances. I had only one ES instance)

Just add one more ES or delete your ES domain and recreate it.