torch.onnx.export() option

torch.onnx.export

Signature:

torch.onnx.export(
    model: 'Union[torch.nn.Module, torch.jit.ScriptModule, torch.jit.ScriptFunction]',
    args: 'Union[Tuple[Any, ...], torch.Tensor]',
    f: 'Union[str, io.BytesIO]',
    export_params: 'bool' = True,
    verbose: 'bool' = False,
    training: '_C_onnx.TrainingMode' = <TrainingMode.EVAL: 0>,
    input_names: 'Optional[Sequence[str]]' = None,
    output_names: 'Optional[Sequence[str]]' = None,
    operator_export_type: '_C_onnx.OperatorExportTypes' = <OperatorExportTypes.ONNX: 0>,
    opset_version: 'Optional[int]' = None,
    do_constant_folding: 'bool' = True,
    dynamic_axes: 'Optional[Union[Mapping[str, Mapping[int, str]], Mapping[str, Sequence[int]]]]' = None,
    keep_initializers_as_inputs: 'Optional[bool]' = None,
    custom_opsets: 'Optional[Mapping[str, int]]' = None,
    export_modules_as_functions: 'Union[bool, Collection[Type[torch.nn.Module]]]' = False,
) -> 'None'

Docstring:
Exports a model into ONNX format.

If model is not a :class:torch.jit.ScriptModule nor a :class:torch.jit.ScriptFunction, this runs model once in order to convert it to a TorchScript graph to be exported (the equivalent of :func:torch.jit.trace). Thus this has the same limited support
for dynamic control flow as :func:torch.jit.trace.

Args:

• model (:class:torch.nn.Module, :class:torch.jit.ScriptModule or :class:torch.jit.ScriptFunction): the model to be exported.

• args (tuple or torch.Tensor):
  1. ONLY A TUPLE OF ARGUMENTS::The tuple should contain model inputs such that model(*args) is a valid invocation of the model. Any non-Tensor arguments will be hard-coded into the exported model; any Tensor arguments will become inputs of the exported model, in the order they occur in the tuple.
  2. args = (x, y, z)
  3. A TENSOR::This is equivalent to a 1-ary tuple of that Tensor.
  4. args = torch.Tensor([1])
  5. A TUPLE OF ARGUMENTS ENDING WITH A DICTIONARY OF NAMED ARGUMENTS::

      x,
      {
          "y": input_y,
          "z": input_z
      }

     )::note::
     If a dictionary is the last element of the args tuple, it will be interpreted as containing named arguments. In order to pass a dict as the last non-keyword arg, provide an empty dict as the last element of the args tuple.
     For example, instead of::
  6. torch.onnx.export( model, ( x, # WRONG: will be interpreted as named arguments {y: z} ), "test.onnx.pb" ) Write:: torch.onnx.export( model, ( x, {y: z}, {} ), "test.onnx.pb" )
  7. All but the last element of the tuple will be passed as non-keyword arguments, and named arguments will be set from the last element. If a named argument is not present in the dictionary, it is assigned the default value, or None if a default value is not provided.
  8. args = (
• args can be structured either as:

• f: a file-like object (such that f.fileno() returns a file descriptor) or a string containing a file name. A binary protocol buffer will be written to this file.

• export_params (bool, default True): if True, all parameters will be exported. Set this to False if you want to export an untrained model.
  In this case, the exported model will first take all of its parameters as arguments, with the ordering as specified by model.state_dict().values()

• verbose (bool, default False): if True, prints a description of the model being exported to stdout. In addition, the final ONNX graph will include the field doc_string from the exported model which mentions the source code locations for model. If True, ONNX exporter logging will be turned on.

• training (enum, default TrainingMode.EVAL):
  • TrainingMode.EVAL: export the model in inference mode.
  • TrainingMode.PRESERVE: export the model in inference mode if model.training is
    False and in training mode if model.training is True.
  • TrainingMode.TRAINING: export the model in training mode. Disables optimizations
    which might interfere with training.
• input_name (list of str, default empty list): names to assign to the input nodes of the graph, in order.

• output_names (list of str, default empty list): names to assign to the output nodes of the graph, in order.

• operator_export_type (enum, default OperatorExportTypes.ONNX):
  • OperatorExportTypes.ONNX: Export all ops as regular ONNX ops
    (in the default opset domain).
  • OperatorExportTypes.ONNX_FALLTHROUGH: Try to convert all ops
    to standard ONNX ops in the default opset domain. If unable to do so
    (e.g. because support has not been added to convert a particular torch op to ONNX),
    fall back to exporting the op into a custom opset domain without conversion. Applies
    to custom ops <https://pytorch.org/tutorials/advanced/torch_script_custom_ops.html>_
    as well as ATen ops. For the exported model to be usable, the runtime must support
    these non-standard ops.
  • OperatorExportTypes.ONNX_ATEN: All ATen ops (in the TorchScript namespace "aten")
    are exported as ATen ops (in opset domain "org.pytorch.aten").
    ATen <https://pytorch.org/cppdocs/#aten>_ is PyTorch's built-in tensor library, so
    this instructs the runtime to use PyTorch's implementation of these ops.

      Models exported this way are probably runnable only by Caffe2.
    
      This may be useful if the numeric differences in implementations of operators are
      causing large differences in behavior between PyTorch and Caffe2 (which is more
      common on untrained models).

  • ::warning::
  • OperatorExportTypes.ONNX_ATEN_FALLBACK: Try to export each ATen op
    (in the TorchScript namespace "aten") as a regular ONNX op. If we are unable to do so
    (e.g. because support has not been added to convert a particular torch op to ONNX),
    fall back to exporting an ATen op. See documentation on OperatorExportTypes.ONNX_ATEN for
    context.
    For example::Assuming aten::triu is not supported in ONNX, this will be exported as::If PyTorch was built with Caffe2 (i.e. with BUILD_CAFFE2=1), then
    Caffe2-specific behavior will be enabled, including special support
    for ops are produced by the modules described in
    Quantization <https://pytorch.org/docs/stable/quantization.html>_.

      Models exported this way are probably runnable only by Caffe2.

  • ::warning::
  • graph(%0 : Float): %1 : Long() = onnx::Constant[value={0}]() # not converted %2 : Float = aten::ATen[operator="triu"](%0, %1) # converted %3 : Float = onnx::Mul(%2, %0) return (%3)
  • graph(%0 : Float): %3 : int = prim::Constant[value=0]() # conversion unsupported %4 : Float = aten::triu(%0, %3) # conversion supported %5 : Float = aten::mul(%4, %0) return (%5)

• opset_version (int, default 14): The version of the default (ai.onnx) opset <https://github.com/onnx/onnx/blob/master/docs/Operators.md> _ to target. Must be >= 7 and <= 16.

• do_constant_folding (bool, default True): Apply the constant-folding optimization.
  Constant-folding will replace some of the ops that have all constant inputs with pre-computed constant nodes.

• dynamic_axes (dict[string, dict[int, string]] or dict[string, list(int)], default empty dict):
  • KEY (str): an input or output name. Each name must also be provided in input_names or
    output_names.
  • VALUE (dict or list): If a dict, keys are axis indices and values are axis names. If a
    list, each element is an axis index.

      class SumModule(torch.nn.Module):
          def forward(self, x):
              return torch.sum(x, dim=1)
    
      torch.onnx.export(
          SumModule(),
          (torch.ones(2, 2),),
          "onnx.pb",
          input_names=["x"],
          output_names=["sum"]
      )

    Produces::While::Produces::
  • input { name: "x" ... shape { dim { dim_param: "my_custom_axis_name" # axis 0 } dim { dim_value: 2 # axis 1 ... output { name: "sum" ... shape { dim { dim_param: "sum_dynamic_axes_1" # axis 0 ...
  • torch.onnx.export( SumModule(), (torch.ones(2, 2),), "onnx.pb", input_names=["x"], output_names=["sum"], dynamic_axes={ # dict value: manually named axes "x": {0: "my_custom_axis_name"}, # list value: automatic names "sum": [0], } )
  • input { name: "x" ... shape { dim { dim_value: 2 # axis 0 } dim { dim_value: 2 # axis 1 ... output { name: "sum" ... shape { dim { dim_value: 2 # axis 0 ...
  • For example::
• By default the exported model will have the shapes of all input and output tensors
  set to exactly match those given in args. To specify axes of tensors as
  dynamic (i.e. known only at run-time), set dynamic_axes to a dict with schema:

• keep_initializers_as_inputs (bool, default None):
  If True, all the initializers (typically corresponding to parameters) in the exported graph will also be added as inputs to the graph.
  If False, then initializers are not added as inputs to the graph, and only the non-parameter inputs are added as inputs.
  This may allow for better optimizations (e.g. constant folding) by backends/runtimes.If None, then the behavior is chosen automatically as follows:
  • If operator_export_type=OperatorExportTypes.ONNX, the behavior is equivalent
    to setting this argument to False.
  • Else, the behavior is equivalent to setting this argument to True.
• If opset_version < 9, initializers MUST be part of graph
  inputs and this argument will be ignored and the behavior will be
  equivalent to setting this argument to True.

• custom_opsets (dict[str, int], default empty dict): A dict with schema:
  • KEY (str): opset domain name
  • VALUE (int): opset version
  • If a custom opset is referenced by model but not mentioned in this dictionary,
    the opset version is set to 1. Only custom opset domain name and version should be
    indicated through this argument.

• export_modules_as_functions (bool or set of type of nn.Module, default False): Flag to enable
  exporting all nn.Module forward calls as local functions in ONNX. Or a set to indicate the
  particular types of modules to export as local functions in ONNX.
  This feature requires opset_version >= 15, otherwise the export will fail. This is because
  opset_version < 15 implies IR version < 8, which means no local function support.
  Module variables will be exported as function attributes. There are two categories of function
  attributes.
  1. Annotated attributes: class variables that have type annotations via
     PEP 526-style <https://www.python.org/dev/peps/pep-0526/#class-and-instance-variable-annotations>_
     will be exported as attributes.
     Annotated attributes are not used inside the subgraph of ONNX local function because
     they are not created by PyTorch JIT tracing, but they may be used by consumers
     to determine whether or not to replace the function with a particular fused kernel.
  2. Inferred attributes: variables that are used by operators inside the module. Attribute names
     will have prefix "inferred::". This is to differentiate from predefined attributes retrieved from
     python module annotations. Inferred attributes are used inside the subgraph of ONNX local function.
     • False (default): export nn.Module forward calls as fine grained nodes.
     • True: export all nn.Module forward calls as local function nodes.
     • Set of type of nn.Module: export nn.Module forward calls as local function nodes,
       only if the type of the nn.Module is found in the set.

Raises:

:class:`torch.onnx.errors.CheckerError`: If the ONNX checker detects an invalid ONNX graph.
:class:`torch.onnx.errors.UnsupportedOperatorError`: If the ONNX graph cannot be exported because it
    uses an operator that is not supported by the exporter.
:class:`torch.onnx.errors.OnnxExporterError`: Other errors that can occur during export.
    All errors are subclasses of :class:`errors.OnnxExporterError`.  

File: 

/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py   

Type:

function

 

---

Example

고정 배치

dummy_input = torch.randn(10, 3, 32, 32, device='cpu')
model.to('cpu')
torch.onnx.export(model, dummy_input, 
                  'resnet_CIFAR10.onnx', 
                  verbose=True,
                  input_names=['input'],
                  output_names=['output'])

그래프 출력

onnx_model = onnx.load('resnet_CIFAR10.onnx')
print(onnx.helper.printable_graph(onnx_model.graph))

graph torch_jit (
  %input[FLOAT, 10x3x32x32]
) initializers (
  %fc.weight[FLOAT, 10x2048]
  %fc.bias[FLOAT, 10]
  %onnx::Conv_497[FLOAT, 64x3x7x7]
  %onnx::Conv_498[FLOAT, 64]
  %onnx::Conv_500[FLOAT, 64x64x1x1]
  %onnx::Conv_501[FLOAT, 64]
  %onnx::Conv_503[FLOAT, 64x64x3x3]
  %onnx::Conv_504[FLOAT, 64]

-> 배치사이즈가 10으로 고정됨

 

다이나믹 배치

dummy_input = torch.randn(10, 3, 32, 32, device='cpu')
model.to('cpu')
torch.onnx.export(model, dummy_input, 
                  'resnet_CIFAR10_dynamic.onnx', 
                  verbose=True,
                  input_names=['input'],
                  output_names=['output'],
                  dynamic_axes={'input' : {0 : 'batch_size'},    # 가변적인 길이를 가진 차원
                                'output' : {0 : 'batch_size'}})

그래프출력

dynamic_model = onnx.load('resnet_CIFAR10_dynamic.onnx')
print(onnx.helper.printable_graph(dynamic_model.graph))

graph torch_jit (
  %input[FLOAT, batch_sizex3x32x32]
) initializers (
  %fc.weight[FLOAT, 10x2048]
  %fc.bias[FLOAT, 10]
  %onnx::Conv_497[FLOAT, 64x3x7x7]
  %onnx::Conv_498[FLOAT, 64]
  %onnx::Conv_500[FLOAT, 64x64x1x1]
  %onnx::Conv_501[FLOAT, 64]
  %onnx::Conv_503[FLOAT, 64x64x3x3]
  %onnx::Conv_504[FLOAT, 64]

-> 배치사이즈가 batch_size로 설정됨