Yodle Gists yodlegists
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| prod_indices = get_prod_indices(client) | |
| print 'Indices to close: ', prod_indices | |
| close_successful = curator.close_indices(client, prod_indices) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Specify your host machines as a list of dictionaries with keys "host" and "port", defaults to localhost:9200 | |
| client = elasticsearch.Elasticsearch(hosts=HOST_LIST) |
yodlegists
/ get_most_recent_snapshot.py
Created
June 18, 2016 02:13
Getting the most recent snapshot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def get_most_recent_snapshot(client): | |
| snapshots = curator.get_snapshots(client, REPOSITORY) | |
| return sorted(snapshots)[-1] |
yodlegists
/ get_prod_indices.py
Created
June 18, 2016 02:11
Getting all indices prefixed with "prod_"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def get_prod_indices(client): | |
| indices = curator.get_indices(client) | |
| prefix_filter = curator.build_filter(kindOf="prefix", value=PROD_PREFIX) | |
| return curator.apply_filter(items=indices, **prefix_filter) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def recover_snapshot(client, snapshot): | |
| payload = { | |
| "ignore_unavailable": True, | |
| "rename_pattern": "(.+)", | |
| "rename_replacement": "prod_$1", | |
| "include_aliases": False, | |
| "index_settings": { | |
| "index.number_of_replicas": 1 | |
| } | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def get_snapshot_indices(client, latest_snapshot): | |
| snap_details = curator.get_snapshot(client, REPOSITORY, latest_snapshot) | |
| return snap_details['snapshots'][0]['indices'] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def get_indices_not_in_snapshot(prod_indices, snapshot_indices): | |
| prefixed_snapshot_indices = ["prod_"+index for index in snapshot_indices] | |
| return list(set(prod_indices).difference(prefixed_snapshot_indices)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import elasticsearch # https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/index.html | |
| import curator # http://curator.readthedocs.io/en/v3.4.1/examples.html | |
| import sys | |
| PROD_PREFIX = "prod_" | |
| REPOSITORY = "Your repository here | |
| HOST_LIST = [{"host": "Friendly-hostname-here", "port": 9200}] | |
| def main(): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curator \ | |
| --host localhost:9200 \ | |
| --timeout 43200 \ | |
| delete snapshots \ | |
| --repository prod_s3_repository \ | |
| --prefix snapshot- \ | |
| --older-than 30 \ | |
| --time-unit days \ | |
| --timestring %Y%m%d%H%M%S |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curator \ | |
| --host localhost:9200 \ | |
| snapshot \ | |
| --repository prod_s3_repository \ | |
| --prefix snapshot- \ | |
| --wait_for_completion true \ | |
| --include_global_state false \ | |
| --request_timeout 43200 \ | |
| indices --all-indices |