This is just a list of helpful techniques I have used for
Unfortunately, I have found that pytorch's unwinder can make mistakes, which makes the stack traces less than useful. As soon as the stack unwinder fails, all remaining frames are omitted.
Compiling with -fno-omit-frame-pointer has not helped in my experience.
| # run like this: `nsys profile -c cudaProfilerApi --cuda-graph-trace=node python nvtx_graph_projection.py` | |
| import torch | |
| device = torch.device("cuda") | |
| @torch.no_grad() | |
| def add_two_numbers(a: torch.Tensor, b: torch.Tensor, out: torch.Tensor): | |
| out.copy_(a) | |
| out.add_(b) |
It takes about 700ns to update each node. $ ./exec_node_set_params Updated 1000 nodes in 0.780 ms (0.780 us per node) Success: destination matches updated source buffer (d_srcB). $ ./exec_node_set_params Updated 1000 nodes in 0.698 ms (0.698 us per node) Success: destination matches updated source buffer (d_srcB). $ ./exec_node_set_params Updated 1000 nodes in 0.723 ms (0.723 us per node) Success: destination matches updated source buffer (d_srcB).
| #include <cstdio> // For fprintf and stderr | |
| #include <cstdlib> // For exit and EXIT_FAILURE | |
| #include <cuda_runtime.h> // For CUDA runtime API functions | |
| #include <thread> // For std::thread | |
| #include <atomic> // For std::atomic | |
| #include <unistd.h> // For sleep function | |
| #include <ostream> | |
| #include <iostream> | |
| #include <nvtx3/nvToolsExt.h> |
| /* | |
| * SPDX-FileCopyrightText: Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |
| * SPDX-License-Identifier: Apache-2.0 | |
| * | |
| * Licensed under the Apache License, Version 2.0 (the "License"); | |
| * you may not use this file except in compliance with the License. | |
| * You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 | |
| * |
| std = {_G = true, package = true, "_VERSION", "arg", "assert", "bit", "collectgarbage", "coroutine", | |
| "debug", "dofile", "error", "gcinfo", "getfenv", "getmetatable", "io", "ipairs", "jit", | |
| "load", "loadfile", "loadstring", "math", "module", "newproxy", "next", "os", "pairs", | |
| "pcall", "print", "rawequal", "rawget", "rawset", "require", "select", "setfenv", | |
| "setmetatable", "string", "table", "tonumber", "tostring", "type", "unpack", "xpcall", | |
| "nn", "torch", "cutorch", "paths", "pl", "image", "cudnn"} |