Dewarp fisheye image

[guidovanschie-nedap]

I have several Axis M3058-PLVE camera's with which I can (among others) output the full fisheye image or a set of 4 dewarped "normal" views. I would like to do this transformation from full fisheye to a dewarped "normal" part of the view myself. I have downloaded the SDK with which this should be possible, but the problem is that the actual functionality is in the compiled Windows DLL. I am not able to use this for my project. Is there open source code (preferably python) or a white-paper available that shows me how to do this?

[github-actions[bot]]

This automatically generated reply acts as a friendly reminder.

Answers to your questions will most often come from the community, from developers like yourself. You will, from time to time, find that Axis employees answers some of the questions, but this is not a guarantee. Think of the discussion forum as a complement to other support channels, not a replacement to any of them.

If your question remains unanswered for a period of time, please revisit it to see whether it can be improved by following the guidelines listed in Axis support guidelines.

[vivekatoffice]

Hi @guidovanschie-nedap ,
As per my understanding, there is no official SDK available for Linux in Python.

I have tried to do this in Python but have not gotten satisfactory results. Just adding the steps here hoping this may help you to get the correct results:

Get the Image sensor and lens details from the datasheet:

Get the Lens parameters using APIs:

Reference: AXIS Dewarping SDK\README.html

Pyhton code (Not working):

import cv2
import numpy as np
import requests
from requests.auth import HTTPDigestAuth
import xml.etree.ElementTree as ET

# Credentials and camera settings
username = "Vivek"
password = "Kumar"
camera_ip = "195.60.68.14"

# HTTP URL to retrieve lens parameters
http_url = f"http://{camera_ip}:80/axis-cgi/lensparams/getradialdistortionparams.cgi?schemaversion=1"

# Retrieve lens parameters with Digest authentication
response = requests.get(http_url, auth=HTTPDigestAuth(username, password))
response.raise_for_status()  # Check if the request was successful

# Print the raw response for debugging
print("Raw Response:")
print(response.text)

# Parse the response XML with namespace handling
root = ET.fromstring(response.text)
namespace = {'ns': 'http://www.axis.com/vapix/http_cgi/lensparams-1'}

# Find coefficient data using namespace
coeffs = root.findall(".//ns:CoefficientData", namespace)

# Extract lens parameters from the response
lens_params = []
for coeff in coeffs:
    coeff_name = coeff.find('ns:CoeffName', namespace).text
    coeff_value = float(coeff.find('ns:CoeffValue', namespace).text)
    lens_params.append(coeff_value)

print("Raw Lens Parameters:", lens_params)

# Check if we have the expected number of parameters
if len(lens_params) < 3:
    raise ValueError("Not enough lens parameters retrieved from the API")

# Resolution height of the current resolution (800x800)
height_of_current_resolution = 800

# Normalize lens parameters
lens_params = [param / height_of_current_resolution for param in lens_params]
print("Normalized Lens Parameters:", lens_params)

# Lens parameters in the format [k1, k2, k3, k4] for fisheye calibration
k1, k2, k3 = lens_params
k4 = 0  # Default value if not provided
D = np.array([k1, k2, k3, k4], dtype=np.float32)

# RTSP URL of the fisheye camera
rtsp_port = "554"
stream_path = "axis-media/media.amp?resolution=800x800"  # Updated stream path
rtsp_url = f"rtsp://{username}:{password}@{camera_ip}:{rtsp_port}/{stream_path}"

# Open the video stream
cap = cv2.VideoCapture(rtsp_url)

if not cap.isOpened():
    print("Error: Could not open video stream.")
    exit()

# Set the resolution explicitly
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 800)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 800)

# Get frame size from video capture
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

print(f"Frame Resolution: {width}x{height}")

# Camera sensor size (1/1.7") in mm
sensor_width_mm = 8.8  # Approximate size for 1/1.7" sensor
sensor_height_mm = 6.6

# Focal length (1.3 mm) in pixels
focal_length_mm = 1.3

# Convert focal length to pixels
fx = (focal_length_mm / sensor_width_mm) * width
fy = (focal_length_mm / sensor_height_mm) * height

# Principal point (cx, cy) set to the center of the image
cx = width / 2
cy = height / 2

# Intrinsic matrix (K)
K = np.array([
    [fx, 0, cx],
    [0, fy, cy],
    [0, 0, 1]
], dtype=np.float32)

print("Intrinsic Matrix (K):")
print(K)

# Compute the new camera matrix
new_K, _ = cv2.getOptimalNewCameraMatrix(K, D, (width, height), 1, (width, height))

# Precompute the undistortion map
map_x, map_y = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3, dtype=np.float32), new_K, (width, height), cv2.CV_16SC2)

while True:
    ret, frame = cap.read()
    if not ret:
        print("Error: Could not read frame.")
        break
    
    # Dewarp the fisheye image
    dewarped_frame = cv2.remap(frame, map_x, map_y, interpolation=cv2.INTER_LINEAR)
    
    # Display the dewarped frame
    cv2.imshow('Dewarped Video Feed', dewarped_frame)
    
    # Break the loop on 'q' key press
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture and close all windows
cap.release()
cv2.destroyAllWindows()

Other references:

Reference1: Fisheye camera image dewarp with OpenCV-Python
Reference2: defisheye

[vivekatoffice]

Adding other code using the defisheye:

There are other settings or parameters also available which you can try with defisheye 😄

import cv2
from defisheye import Defisheye

# Credentials and camera settings
username = "Vivek"
password = "Kumar"
camera_ip = "195.60.68.14"

# RTSP URL of the fisheye camera
rtsp_port = "554"
stream_path = "axis-media/media.amp?resolution=800x800"  # Updated stream path
rtsp_url = f"rtsp://{username}:{password}@{camera_ip}:{rtsp_port}/{stream_path}"

# Open the video stream
cap = cv2.VideoCapture(rtsp_url)

if not cap.isOpened():
    print("Error: Could not open video stream.")
    exit()

# Read the first frame to determine the resolution
ret, frame = cap.read()
if not ret:
    print("Error: Could not read frame.")
    cap.release()
    exit()

# Resize the frame to a fixed size if needed (e.g., 800x800)
resized_frame = cv2.resize(frame, (800, 800))

# Defisheye parameters
dtype = 'linear'
format = 'fullframe'
fov = 180
pfov = 120

# Create Defisheye object and convert the image
obj = Defisheye(resized_frame, dtype=dtype, format=format, fov=fov, pfov=pfov)
final_frame = obj.convert()

# Display the corrected frame
cv2.imshow('Final Dewarped Video Feed', final_frame)

# Wait for a key press to exit
cv2.waitKey(0)

# Release the video capture and close all windows
cap.release()
cv2.destroyAllWindows()

Input Image:

Output Image:

[guidovanschie-nedap]

Hi @vivekatoffice,

Thank you for posting your code! I have also tried both approaches, but was also not completely satisfied with the result. The lines in the defisheye result are still not straight. My guess was also that the coefficients you get from calling the API are indeed general (per lens type, not specific per lens) coefficients that can be used in the polynomial to approximate the lens model. k1, k2 and k3 in formula 6 in https://oulu3dvision.github.io/calibgeneric/Kannala_Brandt_calibration.pdf
Can you confirm that that is correct?
I wanted to try to use that formula in the defisheye module instead of the general models, but I am not sure if that is the way to go. I also am not sure how the center point works to get specific projections with less distortion.
A paper or some working (C++) code would be really appreciated.

[guidovanschie-nedap]

My main question is that I have no clear understanding of how I can make use of the coefficients I get from my camera.
I think I can get good results when I use these, but I will have to know the function to which they belong and also in what unit they are (do I need to normalize with image dimensions, focal length etc.)?

I have this, but I can find no reference on the internet what to do with them:

<LensParamsResponse SchemaVersion="1.0" xsi:schemaLocation="http://www.axis.com/vapix/http_cgi/lensparams-1 http://www.axis.com/vapix/http_cgi/lensparams-1"> <Success> <GetRadialDistortionParamsSuccess> <CoefficientData> <CoeffName>C1</CoeffName> <CoeffValue>113.889694</CoeffValue> </CoefficientData> <CoefficientData> <CoeffName>C2</CoeffName> <CoeffValue>-60.882477</CoeffValue> </CoefficientData> <CoefficientData> <CoeffName>C3</CoeffName> <CoeffValue>751.488831</CoeffValue> </CoefficientData> </GetRadialDistortionParamsSuccess> </Success> </LensParamsResponse>

[vivekatoffice]

You are correct the SDK Readme document only talks about these values used in sample projects. Let's wait for the response from other members if they have performed the de-warping using the Linux project 😞