Skip to content

Instantly share code, notes, and snippets.

@nodebotanist

nodebotanist/01-register-models.sh

Last active November 26, 2025 18:55
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save nodebotanist/c664dc27747a3cafeca3b9a4a85c572a to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save nodebotanist/c664dc27747a3cafeca3b9a4a85c572a to your computer and use it in GitHub Desktop.
Download ZIP
Creating a Semantic Search app on NetApp Instaclustr OpenSearch
Raw
01-register-models.sh
#!/bin/bash
curl -XPOST -u icopensearch:yourpassword "https://your-cluster-url:9200/_plugins/_ml/models/_register?deploy=true" -H 'Content-Type: application/json' --output 'model-register-response.json' -d'
{
"name": "huggingface/sentence-transformers/msmarco-distilbert-base-tas-b",
"version": "1.0.3",
"model_format": "TORCH_SCRIPT"
}
'
Raw
02-get-model-id.sh
#!/bin/bash
curl -XGET -u icopensearch:yourpassword "https://your-cluster-url:9200/_plugins/_ml/tasks/your-task-id" -H 'Content-Type: application/json' --output 'model-registration-check-response.json'
Raw
03-create-ingest-pipeline.sh
#! /bin/bash
curl -XPUT -u icopensearch:yourpassword "https://your-cluster-url:9200/_ingest/pipeline/example-nlp-ingest-pipeline" -H 'Content-Type: application/json' -d'
{
"description": "A Natural Language Processing ingest pipeline",
"processors": [
{
"text_embedding": {
"model_id": "your-model-id",
"field_map": {
"text": "passage_embedding"
}
}
}
]
}
'
Raw
04-create-vector-index.sh
curl -XPUT -u icopensearch:yourpassword --output index-creation-response.json "https://your-cluster-url:9200/example-nlp-index" -H 'Content-Type: application/json' -d'
{
"settings": {
"index.knn": true,
"default_pipeline": "example-nlp-ingest-pipeline"
},
"mappings": {
"properties": {
"id": {
"type": "text"
},
"passage_embedding": {
"type": "knn_vector",
"dimension": 768,
"space_type": "l2"
},
"text": {
"type": "text"
}
}
}
}
'
Raw
05-ingest-data.sh
#! /bin/bash
curl -XPUT -u icopensearch:yourpassword --output document1-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/1" -H 'Content-Type: application/json' -d'
{
"text": "Spanning the years 1945 to 1955, a chronicle of the fictional Italian-American Corleone crime family. When organized crime family patriarch, Vito Corleone barely survives an attempt on his life, his youngest son, Michael steps in to take care of the would-be killers, launching a campaign of bloody revenge.",
"id": "the-godfather"
}
'
curl -XPUT -u icopensearch:yourpassword --output document2-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/2" -H 'Content-Type: application/json' -d'
{
"text": "Framed in the 1940s for the double murder of his wife and her lover, upstanding banker Andy Dufresne begins a new life at the Shawshank prison, where he puts his accounting skills to work for an amoral warden. During his long stretch in prison, Dufresne comes to be admired by the other inmates -- including an older prisoner named Red -- for his integrity and unquenchable sense of hope.",
"id": "the-shawshank-redemption"
}
'
curl -XPUT -u icopensearch:yourpassword --output document3-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/3" -H 'Content-Type: application/json' -d'
{
"text": "70-year-old Malena gets a second chance at life when she magically turns into her 22-year-old self. Now, posing as \"Maria\" to hide her true identity, she becomes the lead singer of her grandsons band and tries to recover her dream of singing, which she had to give up at some point.",
"id": "cuando-sea-joven"
}
'
curl -XPUT -u icopensearch:yourpassword --output document4-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/4" -H 'Content-Type: application/json' -d'
{
"text": "In the continuing saga of the Corleone crime family, a young Vito Corleone grows up in Sicily and in 1910s New York. In the 1950s, Michael Corleone attempts to expand the family business into Las Vegas, Hollywood and Cuba.",
"id": "the-godfather-part-ii"
}
'
curl -XPUT -u icopensearch:yourpassword --output document5-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/5" -H 'Content-Type: application/json' -d'
{
"text": "The true story of how businessman Oskar Schindler saved over a thousand Jewish lives from the Nazis while they worked as slaves in his factory during World War II.",
"id": "schindlers-list"
}
'
curl -XPUT -u icopensearch:yourpassword --output document6-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/6" -H 'Content-Type: application/json' -d'
{
"text": "The defense and the prosecution have rested and the jury is filing into the jury room to decide if a young Spanish-American is guilty or innocent of murdering his father. What begins as an open and shut case soon becomes a mini-drama of each of the jurors prejudices and preconceptions about the trial, the accused, and each other.",
"id": "12-angry-men"
}
'
curl -XPUT -u icopensearch:yourpassword --output document7-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/7" -H 'Content-Type: application/json' -d'
{
"text": "Batman raises the stakes in his war on crime. With the help of Lt. Jim Gordon and District Attorney Harvey Dent, Batman sets out to dismantle the remaining criminal organizations that plague the streets. The partnership proves to be effective, but they soon find themselves prey to a reign of chaos unleashed by a rising criminal mastermind known to the terrified citizens of Gotham as the Joker.",
"id": "the-dark-knight"
}
'
curl -XPUT -u icopensearch:yourpassword --output document8-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/8" -H 'Content-Type: application/json' -d'
{
"text": "A supernatural tale set on death row in a Southern prison, where gentle giant John Coffey possesses the mysterious power to heal peoples ailments. When the cell blocks head guard, Paul Edgecomb, recognizes Coffeys miraculous gift, he tries desperately to help stave off the condemned mans execution.",
"id": "the-green-mile"
}
'
curl -XPUT -u icopensearch:yourpassword --output document9-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/9" -H 'Content-Type: application/json' -d'
{
"text": "The unlikely friendship of a boy, a mole, a fox and a horse traveling together in the boys search for home.",
"id": "the-boy-the-mole-the-fox-and-the-horse"
}
'
curl -XPUT -u icopensearch:yourpassword --output document10-creation-response.json "https://your-cluster-url:9200/example-nlp-index/_doc/10" -H 'Content-Type: application/json' -d'
{
"text": "A man with a low IQ has accomplished great things in his life and been present during significant historic events—in each case, far exceeding what anyone imagined he could do. But despite all he has achieved, his one true love eludes him.",
"id": "forrest-gump"
}
'
Raw
06-neural-search.sh
#! /bin/bash
curl -XGET -u icopensearch:yourpassword --output neural-search-response.json "https://your-cluster-url:9200/my-nlp-index/_search" -H 'Content-Type: application/json' -d'
{
"_source": {
"excludes": [
"passage_embedding"
]
},
"query": {
"neural": {
"passage_embedding": {
"query_text": "adversity overcome through determination",
"model_id": "your-model-id",
"k": 5
}
}
}
}
'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment