Content :
(Internal Tranining Material)
Usually the first step in performance optimization is to do profiling, e.g. to identify performance hotspots of a workload. This gist tells basic knowledge of performance profiling on PyTorch, you will get:
- How to find the bottleneck operator?
- How to trace source file of a particular operator?
- How do I indentify threading issues? (oversubscription)
- How do I tell a specific operator is running efficiently or not?
This tutorial takes one of my recent projects - pssp-transformer as an example to guide you through path of PyTorch CPU peformance optimization. Focus will be on Part 1 & Part 2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Working example for my blog post at: | |
| # https://danijar.github.io/structuring-your-tensorflow-models | |
| import functools | |
| import tensorflow as tf | |
| from tensorflow.examples.tutorials.mnist import input_data | |
| def doublewrap(function): | |
| """ | |
| A decorator decorator, allowing to use the decorator to be used without |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| % From http://phaseportrait.blogspot.com/2008/06/sinc-interpolation-in-matlab.html | |
| % Ideally "resamples" x vector from s to u by sinc interpolation | |
| function y = sinc_interp(x,s,u) | |
| % Interpolates x sampled sampled at "s" instants | |
| % Output y is sampled at "u" instants ("u" for "upsampled") | |
| % (EXPECTS x, s, and u to be ROW VECTORS!!) | |
| % Find the period of the undersampled signal | |
| T = s(2)-s(1); |
