Skip to content

Instantly share code, notes, and snippets.

@l1m2p3

l1m2p3/extract.py

Last active January 7, 2018 22:59
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save l1m2p3/ee7383f823f4144d20f2e7920caefbf4 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save l1m2p3/ee7383f823f4144d20f2e7920caefbf4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
extract.py
import torch
import numpy
import pickle
# change this to your own files' name
dataset_file = 'word2vec.sst-1.pt' # file to load from
wordindex_file = 'wordindex.pkl' # file to save to
indexvec_file = 'indexvec.npy' # file to save to
# load dataset
stoi, vectors, dim = torch.load(dataset_file)
indexvec = vectors.numpy()
wordindex = stoi
# save lookups to file
with open(wordindex_file, 'wb') as f:
pickle.dump(wordindex, f, pickle.HIGHEST_PROTOCOL)
numpy.save(indexvec_file, indexvec)
# load lookups from file
# with open(wordindex_file, 'rb') as f:
# wordindex = pickle.load(f)
# indexvec = numpy.load(indexvec_file)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment