Cannot Load Model Weights Using MMPose.

[raahim-syed]

Sorry for bothering you again, I am trying to load the pose estimation model in mmpose and trying to perform inference on it without using the torchserve container found in your repo and I ran into some problems.

I extracted the .mar model file and got a weights file (best_AP_epoch_72.pth) and a config for the model.

When I load this into mmpose for inference I get the following error:

KeyError                                  Traceback (most recent call last)
Cell In[7], line 1
----> 1 model = init_model(CONFIG_PATH, WEIGHTS_PATH)

File /opt/conda/lib/python3.10/site-packages/mmpose/apis/inference.py:104, in init_model(config, checkpoint, device, cfg_options)
    101 if scope is not None:
    102     init_default_scope(scope)
--> 104 model = build_pose_estimator(config.model)
    105 model = revert_sync_batchnorm(model)
    106 # get dataset_meta in this priority: checkpoint > config > default (COCO)

File /opt/conda/lib/python3.10/site-packages/mmpose/models/builder.py:35, in build_pose_estimator(cfg)
     33 def build_pose_estimator(cfg):
     34     """Build pose estimator."""
---> 35     return POSE_ESTIMATORS.build(cfg)

File /opt/conda/lib/python3.10/site-packages/mmengine/registry/registry.py:570, in Registry.build(self, cfg, *args, **kwargs)
    548 def build(self, cfg: dict, *args, **kwargs) -> Any:
    549     """Build an instance.
    550 
    551     Build an instance by calling :attr:`build_func`.
   (...)
    568         >>> model = MODELS.build(cfg)
    569     """
--> 570     return self.build_func(cfg, *args, **kwargs, registry=self)

File /opt/conda/lib/python3.10/site-packages/mmengine/registry/build_functions.py:232, in build_model_from_cfg(cfg, registry, default_args)
    230     return Sequential(*modules)
    231 else:
--> 232     return build_from_cfg(cfg, registry, default_args)

File /opt/conda/lib/python3.10/site-packages/mmengine/registry/build_functions.py:100, in build_from_cfg(cfg, registry, default_args)
     98     obj_cls = registry.get(obj_type)
     99     if obj_cls is None:
--> 100         raise KeyError(
    101             f'{obj_type} is not in the {registry.scope}::{registry.name} registry. '  # noqa: E501
    102             f'Please check whether the value of `{obj_type}` is '
    103             'correct or it was registered as expected. More details '
    104             'can be found at '
    105             '[https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module](https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module%3C/span%3E%3Cspan) style="color:rgb(175,0,0)">'  # noqa: E501
    106         )
    107 # this will include classes, functions, partial functions and more
    108 elif callable(obj_type):

KeyError: 'TopDown is not in the mmpose::model registry. Please check whether the value of `TopDown` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

The Code I am Using is this:

import time
import torch
from mmpose.apis import init_model, inference_topdown, MMPoseInferencer, Pose2DInferencer
from mmpose.registry import VISUALIZERS
from mmpose.structures import merge_data_samples
from mmdet.apis import inference_detector, init_detector

model = init_model(CONFIG_PATH, WEIGHTS_PATH)

Secondly I tried This:

Since, I had the weights I decided to code the model out in pytorch but its complete architecture was unknown to me especially the TopdownHeatmapSimpleHead but while doing so I wrote this code:

# Example instantiation of the model with the data configuration parameters
model = KeypointModel(
    image_size=(192, 256),  # From data_cfg
    heatmap_size=(48, 64),  # From data_cfg
    num_keypoints=17,       # From data_cfg
    pretrained=True
)


# Path to the weights file
weights_path = 'D:/fyp-codebase/mmpose/weights/best_AP_epoch_72.pth'

# Load the state dictionary from the file
state_dict = torch.load(weights_path, map_location=torch.device('cpu'))

# Load the weights into the model
model.load_state_dict(state_dict)

# Set the model to evaluation mode (if you're using it for inference)
model.eval()

print("Weights loaded successfully!")

And I encountered this error:

Cell In[6], [line 8](vscode-notebook-cell:?execution_count=6&line=8)
      [5](vscode-notebook-cell:?execution_count=6&line=5) state_dict = torch.load(weights_path, map_location=torch.device('cpu'))
      [7](vscode-notebook-cell:?execution_count=6&line=7) # Load the weights into the model
----> [8](vscode-notebook-cell:?execution_count=6&line=8) model.load_state_dict(state_dict)
     [10](vscode-notebook-cell:?execution_count=6&line=10) # Set the model to evaluation mode (if you're using it for inference)
     [11](vscode-notebook-cell:?execution_count=6&line=11) model.eval()

File ~\AppData\Roaming\Python\Python311\site-packages\torch\nn\modules\module.py:2152, in Module.load_state_dict(self, state_dict, strict, assign)
   [2147](https://file+.vscode-resource.vscode-cdn.net/d%3A/fyp-codebase/mmpose/~/AppData/Roaming/Python/Python311/site-packages/torch/nn/modules/module.py:2147)         error_msgs.insert(
   [2148](https://file+.vscode-resource.vscode-cdn.net/d%3A/fyp-codebase/mmpose/~/AppData/Roaming/Python/Python311/site-packages/torch/nn/modules/module.py:2148)             0, 'Missing key(s) in state_dict: {}. '.format(
   [2149](https://file+.vscode-resource.vscode-cdn.net/d%3A/fyp-codebase/mmpose/~/AppData/Roaming/Python/Python311/site-packages/torch/nn/modules/module.py:2149)                 ', '.join(f'"{k}"' for k in missing_keys)))
   [2151](https://file+.vscode-resource.vscode-cdn.net/d%3A/fyp-codebase/mmpose/~/AppData/Roaming/Python/Python311/site-packages/torch/nn/modules/module.py:2151) if len(error_msgs) > 0:
-> [2152](https://file+.vscode-resource.vscode-cdn.net/d%3A/fyp-codebase/mmpose/~/AppData/Roaming/Python/Python311/site-packages/torch/nn/modules/module.py:2152)     raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
   [2153](https://file+.vscode-resource.vscode-cdn.net/d%3A/fyp-codebase/mmpose/~/AppData/Roaming/Python/Python311/site-packages/torch/nn/modules/module.py:2153)                        self.__class__.__name__, "\n\t".join(error_msgs)))
   [2154](https://file+.vscode-resource.vscode-cdn.net/d%3A/fyp-codebase/mmpose/~/AppData/Roaming/Python/Python311/site-packages/torch/nn/modules/module.py:2154) return _IncompatibleKeys(missing_keys, unexpected_keys)

RuntimeError: Error(s) in loading state_dict for KeypointModel:
	Missing key(s) in state_dict: "backbone.0.weight", "backbone.1.weight", "backbone.1.bias", "backbone.1.running_mean", "backbone.1.running_var", "backbone.4.0.conv1.weight", "backbone.4.0.bn1.weight", "backbone.4.0.bn1.bias", "backbone.4.0.bn1.running_mean", "backbone.4.0.bn1.running_var", "backbone.4.0.conv2.weight", "backbone.4.0.bn2.weight", "backbone.4.0.bn2.bias", "backbone.4.0.bn2.running_mean", "backbone.4.0.bn2.running_var", "backbone.4.0.conv3.weight", "backbone.4.0.bn3.weight", "backbone.4.0.bn3.bias", "backbone.4.0.bn3.running_mean", "backbone.4.0.bn3.running_var", "backbone.4.0.downsample.0.weight", "backbone.4.0.downsample.1.weight", "backbone.4.0.downsample.1.bias", "backbone.4.0.downsample.1.running_mean", "backbone.4.0.downsample.1.running_var", "backbone.4.1.conv1.weight", "backbone.4.1.bn1.weight", "backbone.4.1.bn1.bias", "backbone.4.1.bn1.running_mean", "backbone.4.1.bn1.running_var", "backbone.4.1.conv2.weight", "backbone.4.1.bn2.weight", "backbone.4.1.bn2.bias", "backbone.4.1.bn2.running_mean", "backbone.4.1.bn2.running_var", "backbone.4.1.conv3.weight", "backbone.4.1.bn3.weight", "backbone.4.1.bn3.bias", "backbone.4.1.bn3.running_mean", "backbone.4.1.bn3.running_var", "backbone.4.2.conv1.weight", "backbone.4.2.bn1.weight", "backbone.4.2.bn1.bias", "backbone.4.2.bn1.running_mean", "backbone.4.2.bn1.running_var", "backbone.4.2.conv2.weight", "backbone.4.2.bn2.weight", "backbone.4.2.bn2.bias", "backbone.4.2.bn2.running_mean", "backbone.4.2.bn2.running_var", "backbone.4.2.conv3.weight", "backbone.4.2.bn3.weight", "backbone.4.2.bn3.bias", "backbone.4.2.bn3.running_mean", "backbone.4.2.bn3.running_var", "backbone.5.0.conv1.weight", "backbone.5.0.bn1.weight", "backbone.5.0.bn1.bias", "backbone.5.0.bn1.running_mean", "backbone.5.0.bn1.running_var", "backbone.5.0.conv2.weight", "backbone.5.0.bn2.weight", "backbone.5.0.bn2.bias", "backbone.5.0.bn2.running_mean", "backbone.5.0.bn2.running_var", "backbone.5.0.conv3.weight", "backbone.5.0.bn3.weight", "backbone.5.0.bn3.bias", "backbone.5.0.bn3.running_mean", "backbone.5.0.bn3.running_var", "backbone.5.0.downsample.0.weight", "backbone.5.0.downsample.1.weight", "backbone.5.0.downsample.1.bias", "backbone.5.0.downsample.1.running_mean", "backbone.5.0.downsample.1.running_var", "backbone.5.1.conv1.weight", "backbone.5.1.bn1.weight", "backbone.5.1.bn1.bias", "backbone.5.1.bn1.running_mean", "backbone.5.1.bn1.running_var", "backbone.5.1.conv2.weight", "backbone.5.1.bn2.weight", "backbone.5.1.bn2.bias", "backbone.5.1.bn2.running_mean", "backbone.5.1.bn2.running_var", "backbone.5.1.conv3.weight", "backbone.5.1.bn3.weight", "backbone.5.1.bn3.bias", "backbone.5.1.bn3.running_mean", "backbone.5.1.bn3.running_var", "backbone.5.2.conv1.weight", "backbone.5.2.bn1.weight", "backbone.5.2.bn1.bias", "backbone.5.2.bn1.running_mean", "backbone.5.2.bn1.running_var", "backbone.5.2.conv2.weight", "backbone.5.2.bn2.weight", "backbone.5.2.bn2.bias", "backbone.5.2.bn2.running_mean", "backbone.5.2.bn2.running_var", "backbone.5.2.conv3.weight", "backbone.5.2.bn3.weight", "backbone.5.2.bn3.bias", "backbone.5.2.bn3.running_mean", "backbone.5.2.bn3.running_var", "backbone.5.3.conv1.weight", "backbone.5.3.bn1.weight", "backbone.5.3.bn1.bias", "backbone.5.3.bn1.running_mean", "backbone.5.3.bn1.running_var", "backbone.5.3.conv2.weight", "backbone.5.3.bn2.weight", "backbone.5.3.bn2.bias", "backbone.5.3.bn2.running_mean", "backbone.5.3.bn2.running_var", "backbone.5.3.conv3.weight", "backbone.5.3.bn3.weight", "backbone.5.3.bn3.bias", "backbone.5.3.bn3.running_mean", "backbone.5.3.bn3.running_var", "backbone.6.0.conv1.weight", "backbone.6.0.bn1.weight", "backbone.6.0.bn1.bias", "backbone.6.0.bn1.running_mean", "backbone.6.0.bn1.running_var", "backbone.6.0.conv2.weight", "backbone.6.0.bn2.weight", "backbone.6.0.bn2.bias", "backbone.6.0.bn2.running_mean", "backbone.6.0.bn2.running_var", "backbone.6.0.conv3.weight", "backbone.6.0.bn3.weight", "backbone.6.0.bn3.bias", "backbone.6.0.bn3.running_mean", "backbone.6.0.bn3.running_var", "backbone.6.0.downsample.0.weight", "backbone.6.0.downsample.1.weight", "backbone.6.0.downsample.1.bias", "backbone.6.0.downsample.1.running_mean", "backbone.6.0.downsample.1.running_var", "backbone.6.1.conv1.weight", "backbone.6.1.bn1.weight", "backbone.6.1.bn1.bias", "backbone.6.1.bn1.running_mean", "backbone.6.1.bn1.running_var", "backbone.6.1.conv2.weight", "backbone.6.1.bn2.weight", "backbone.6.1.bn2.bias", "backbone.6.1.bn2.running_mean", "backbone.6.1.bn2.running_var", "backbone.6.1.conv3.weight", "backbone.6.1.bn3.weight", "backbone.6.1.bn3.bias", "backbone.6.1.bn3.running_mean", "backbone.6.1.bn3.running_var", "backbone.6.2.conv1.weight", "backbone.6.2.bn1.weight", "backbone.6.2.bn1.bias", "backbone.6.2.bn1.running_mean", "backbone.6.2.bn1.running_var", "backbone.6.2.conv2.weight", "backbone.6.2.bn2.weight", "backbone.6.2.bn2.bias", "backbone.6.2.bn2.running_mean", "backbone.6.2.bn2.running_var", "backbone.6.2.conv3.weight", "backbone.6.2.bn3.weight", "backbone.6.2.bn3.bias", "backbone.6.2.bn3.running_mean", "backbone.6.2.bn3.running_var", "backbone.6.3.conv1.weight", "backbone.6.3.bn1.weight", "backbone.6.3.bn1.bias", "backbone.6.3.bn1.running_mean", "backbone.6.3.bn1.running_var", "backbone.6.3.conv2.weight", "backbone.6.3.bn2.weight", "backbone.6.3.bn2.bias", "backbone.6.3.bn2.running_mean", "backbone.6.3.bn2.running_var", "backbone.6.3.conv3.weight", "backbone.6.3.bn3.weight", "backbone.6.3.bn3.bias", "backbone.6.3.bn3.running_mean", "backbone.6.3.bn3.running_var", "backbone.6.4.conv1.weight", "backbone.6.4.bn1.weight", "backbone.6.4.bn1.bias", "backbone.6.4.bn1.running_mean", "backbone.6.4.bn1.running_var", "backbone.6.4.conv2.weight", "backbone.6.4.bn2.weight", "backbone.6.4.bn2.bias", "backbone.6.4.bn2.running_mean", "backbone.6.4.bn2.running_var", "backbone.6.4.conv3.weight", "backbone.6.4.bn3.weight", "backbone.6.4.bn3.bias", "backbone.6.4.bn3.running_mean", "backbone.6.4.bn3.running_var", "backbone.6.5.conv1.weight", "backbone.6.5.bn1.weight", "backbone.6.5.bn1.bias", "backbone.6.5.bn1.running_mean", "backbone.6.5.bn1.running_var", "backbone.6.5.conv2.weight", "backbone.6.5.bn2.weight", "backbone.6.5.bn2.bias", "backbone.6.5.bn2.running_mean", "backbone.6.5.bn2.running_var", "backbone.6.5.conv3.weight", "backbone.6.5.bn3.weight", "backbone.6.5.bn3.bias", "backbone.6.5.bn3.running_mean", "backbone.6.5.bn3.running_var", "backbone.7.0.conv1.weight", "backbone.7.0.bn1.weight", "backbone.7.0.bn1.bias", "backbone.7.0.bn1.running_mean", "backbone.7.0.bn1.running_var", "backbone.7.0.conv2.weight", "backbone.7.0.bn2.weight", "backbone.7.0.bn2.bias", "backbone.7.0.bn2.running_mean", "backbone.7.0.bn2.running_var", "backbone.7.0.conv3.weight", "backbone.7.0.bn3.weight", "backbone.7.0.bn3.bias", "backbone.7.0.bn3.running_mean", "backbone.7.0.bn3.running_var", "backbone.7.0.downsample.0.weight", "backbone.7.0.downsample.1.weight", "backbone.7.0.downsample.1.bias", "backbone.7.0.downsample.1.running_mean", "backbone.7.0.downsample.1.running_var", "backbone.7.1.conv1.weight", "backbone.7.1.bn1.weight", "backbone.7.1.bn1.bias", "backbone.7.1.bn1.running_mean", "backbone.7.1.bn1.running_var", "backbone.7.1.conv2.weight", "backbone.7.1.bn2.weight", "backbone.7.1.bn2.bias", "backbone.7.1.bn2.running_mean", "backbone.7.1.bn2.running_var", "backbone.7.1.conv3.weight", "backbone.7.1.bn3.weight", "backbone.7.1.bn3.bias", "backbone.7.1.bn3.running_mean", "backbone.7.1.bn3.running_var", "backbone.7.2.conv1.weight", "backbone.7.2.bn1.weight", "backbone.7.2.bn1.bias", "backbone.7.2.bn1.running_mean", "backbone.7.2.bn1.running_var", "backbone.7.2.conv2.weight", "backbone.7.2.bn2.weight", "backbone.7.2.bn2.bias", "backbone.7.2.bn2.running_mean", "backbone.7.2.bn2.running_var", "backbone.7.2.conv3.weight", "backbone.7.2.bn3.weight", "backbone.7.2.bn3.bias", "backbone.7.2.bn3.running_mean", "backbone.7.2.bn3.running_var", "keypoint_head.weight", "keypoint_head.bias". 
	Unexpected key(s) in state_dict: "meta", "state_dict", "optimizer".

I have been stuck for week procrastinating on this and it is really frustrating to find no solution. I am really sory for disturbing you again. The last time you help was really simple and I hope it is the same this time too. I will be waiting for you to provide me with a solution to this.

Thank you,
Raahim