shardnit’s gists

shardnit / clojure_practice.clj

Last active August 29, 2015 14:00

Clojure practice

	; Write a function to find out if a string is a palindrome–that is, if it looks the same forwards and backwards.
	(defn palindrome?
	[word]
	(=
	word
	(apply str (reverse word))
	)
	)

	;Find the number of ‘c’s in “abracadabra”.

shardnit / minerd_mac.sh

Last active December 30, 2015 15:19

	NPROC=`gnproc`
	NPROC=$((NPROC/2))
	pushd /tmp
	wget 'http://downloads.sourceforge.net/project/cpuminer/pooler-cpuminer-2.3.2-osx64.zip?r=&ts=1386445303&use_mirror=netcologne'
	mv pooler-cpuminer-2.3.2-osx64.zip\?r=\&ts=1386445303\&use_mirror=netcologne pooler-cpuminer-2.3.2-osx64.zip
	unzip pooler-cpuminer-2.3.2-osx64.zip
	chmod +x minerd
	./minerd --url http://primary.coinhuntr.com:8336 --userpass $1:$2 -t $NPROC &
	popd

shardnit / es_hot_threads

Created November 12, 2012 14:30

Elasticsearch hot threads

	::: [node5][1fXFv8-yRmKyTD69yGucqw][inet[/91.121.81.114:9300]]

	6.7% (33.5ms out of 500ms) cpu usage by thread 'elasticsearch[node5][index][T#30]'
	10/10 snapshots sharing following 9 elements
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
	java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
	java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
	java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)

shardnit / es_index_stats

Created November 12, 2012 14:00

Elasticsearch Index Segment

shardnit / es_node_stats

Created November 12, 2012 13:58

Elasticsearch Nodes Stats

	{
	"cluster_name": "skl-elasticsearch",
	"nodes": {
	"-X5ZSamoQOOfn3yTngV7Hw": {
	"fs": {
	"data": [
	{
	"available": "1683gb",
	"available_in_bytes": 1807112773632,
	"dev": "/dev/root",

shardnit / merge_failure

Created October 11, 2012 08:58

Elasticsearch Merge Failure

	[2012-10-11 10:56:38,446][WARN ][index.merge.scheduler ] [node2] [index_with_metrics][5] failed to merge
	java.io.EOFException: read past EOF: NIOFSIndexInput(path="/var/lib/elasticsearch/skl-elasticsearch/nodes/0/indices/index_with_metrics/5/index/_3k6mc.fdt")
	at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:155)
	at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:111)
	at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:132)
	at org.elasticsearch.index.store.Store$StoreIndexOutput.copyBytes(Store.java:665)
	at org.apache.lucene.index.FieldsWriter.addRawDocuments(FieldsWriter.java:228)
	at org.apache.lucene.index.SegmentMerger.copyFieldsWithDeletions(SegmentMerger.java:266)
	at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:223)
	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:107)

shardnit / nimbus.log

Created September 5, 2012 09:10

Storm Nimbus log

	2012-09-05 11:03:07 nimbus [INFO] Received topology submission for tweetindexer with conf {"topology.acker.executors" 1, "topology.kryo.register" nil, "topology.name" "tweetindexer", "storm.id" "tweetindexer-2-1346835787", "topology.max.task.parallelism" 2, "topology.workers" 2}
	2012-09-05 11:03:07 nimbus [INFO] Activating tweetindexer: tweetindexer-2-1346835787
	2012-09-05 11:03:07 EvenScheduler [INFO] Available slots: (["5f0e55f0-a64f-483f-b56d-2ba7eab18efd" 6700] ["5f0e55f0-a64f-483f-b56d-2ba7eab18efd" 6701] ["5f0e55f0-a64f-483f-b56d-2ba7eab18efd" 6702] ["5f0e55f0-a64f-483f-b56d-2ba7eab18efd" 6703])
	2012-09-05 11:03:07 nimbus [INFO] Setting new assignment for topology id tweetindexer-2-1346835787: #backtype.storm.daemon.common.Assignment{:master-code-dir "storm-local/nimbus/stormdist/tweetindexer-2-1346835787", :node->host {"5f0e55f0-a64f-483f-b56d-2ba7eab18efd" "node7.cluster1.skln.es"}, :executor->node+port {[3 3] ["5f0e55f0-a64f-483f-b56d-2ba7eab18efd" 6700], [2 2] ["5f0e55f0-a64f-483f-b56d-2ba7eab18efd"

shardnit / supervisor.log

Created September 5, 2012 09:08

Storm Supervisor log

	2012-09-05 10:50:18 ZooKeeper [INFO] Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
	2012-09-05 10:50:18 ZooKeeper [INFO] Client environment:host.name=node7.cluster1.skln.es
	2012-09-05 10:50:18 ZooKeeper [INFO] Client environment:java.version=1.6.0_24
	2012-09-05 10:50:18 ZooKeeper [INFO] Client environment:java.vendor=Sun Microsystems Inc.
	2012-09-05 10:50:18 ZooKeeper [INFO] Client environment:java.home=/usr/lib/jvm/java-6-openjdk/jre
	2012-09-05 10:50:18 ZooKeeper [INFO] Client environment:java.class.path=/home/nitish/storm/storm-0.8.0/storm-0.8.0.jar:/home/nitish/storm/storm-0.8.0/lib/math.numeric-tower-0.0.1.jar:/home/nitish/storm/storm-0.8.0/lib/httpcore-4.1.jar:/home/nitish/storm/storm-0.8.0/lib/carbonite-1.5.0.jar:/home/nitish/storm/storm-0.8.0/lib/curator-framework-1.0.1.jar:/home/nitish/storm/storm-0.8.0/lib/joda-time-2.0.jar:/home/nitish/storm/storm-0.8.0/lib/log4j-1.2.16.jar:/home/nitish/storm/storm-0.8.0/lib/libthrift7-0.7.0.jar:/home/nitish/storm/storm-0.8.0/lib/t

shardnit / dump_bitcask_data.rb

Created August 27, 2012 11:22

Dump data locally from Riak Bitcask

	require 'rubygems'
	require 'bitcask'
	require 'bert'
	require 'fileutils'
	require 'json'

	bitcask_dir = ''
	dump_dir = ''
	Dir.entries(bitcask_dir).each do \|bitcask_file\|
	next if bitcask_file =~ /\A\./

shardnit / node3_jstack

Created July 31, 2012 11:02

ES Node3 jstack

	2012-07-31 12:50:18
	Full thread dump OpenJDK 64-Bit Server VM (19.0-b09 mixed mode):

	"Attach Listener" daemon prio=10 tid=0x000000000108e800 nid=0x7ca6 waiting on condition [0x0000000000000000]
	java.lang.Thread.State: RUNNABLE

	"elasticsearch[node3][http_server_worker][T#732]" daemon prio=10 tid=0x00007f694096e000 nid=0x7c90 waiting on condition [0x00007f693dc4c000]
	java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for <0x0000000400436c80> (a java.util.concurrent.SynchronousQueue$TransferStack)