TeslaFi Data Import Error

[sdmbtrumpet]

I am hoping someone may be able to point me in the right direction. I have about 1.5 years worth of data that I'm attempting to import into TeslaMate that have been downloaded from TeslaFi. I have modified the files to add a vehicle ID using the online CSV editor linked in #3158 (comment) to prevent inadvertent Excel changes to the date field. The first four months import with no issue, and half of the fifth month imports before throwing an error. The error is:

{%Postgrex.Error{ message: nil, postgres: %{ code: :numeric_value_out_of_range, line: "7979", message: "numeric field overflow", file: "numeric.c", unknown: "ERROR", severity: "ERROR", detail: "A field with precision 9, scale 6 must round to an absolute value less than 10^3.", pg_code: "22003", routine: "apply_typmod" }, connection_id: 109, query: nil }, [ {Ecto.Adapters.SQL, :raise_sql_call_error, 1, [ file: ~c"lib/ecto/adapters/sql.ex", line: 1096, error_info: %{module: Exception} ]}, {Ecto.Repo.Schema, :apply, 4, [file: ~c"lib/ecto/repo/schema.ex", line: 834]}, {Ecto.Repo.Schema, :"-do_insert/4-fun-3-", 15, [file: ~c"lib/ecto/repo/schema.ex", line: 415]}, {TeslaMate.Vehicles.Vehicle, :"-handle_event/4-fun-9-", 3, [file: ~c"lib/teslamate/vehicles/vehicle.ex", line: 1049]}, {Ecto.Adapters.SQL, :"-checkout_or_transaction/4-fun-0-", 3, [file: ~c"lib/ecto/adapters/sql.ex", line: 1400]}, {DBConnection, :run, 3, [file: ~c"lib/db_connection.ex", line: 959]}, {TeslaMate.Vehicles.Vehicle, :handle_event, 4, [file: ~c"lib/teslamate/vehicles/vehicle.ex", line: 1047]}, {:gen_statem, :loop_state_callback, 11, [file: ~c"gen_statem.erl", line: 1397]} ]}

Can someone please point me in the right direction? I do not see any differences in the data from line 7978 to 7979 that would cause this issue. Any help is appreciated.

[Xapier]

Can you tell wich files from TeslaFi have problems during the import or some more error messages (i.e. which column is causing the error)? I noticed that TeslaFi exports changed in the Nov 2024 file as battery_level is now decimal instead integer. All files from november 2024 and later are getting import errors. There is already an issue for this

#4477

[sdmbtrumpet]

My files being failing to import starting with Oct 2023. I'm not super familiar with the database queries and things, so I can't give a definitive answer on which column is throwing the error. I posted the error received from the web import page above if that is able to provide any insight.

[Xapier]

This is strange and seems to be a general problem. Could you post the lines causing the error messages?

I don’t have any problems importing any data from TeslaFi before 11/24

[sdmbtrumpet]

58627,2023-10-16 10:37:03,,,1,Oreo,,,,[VIN],,,0,,online,,0.0,32,1,,,0,,230.17,0,100,32,Engaged,,0,0,,85,,,,86,47.12,0,,,230.17,,Disconnected,0,,,,205.77,0.0,2,32,0,213.0,,213.0,20.2,-94.732509,3,,38.958292,,,0,,0,22.0,0,22.0,11.0,0,0,,,0,0,0,0,,,,,,,0,,0,,,0,8150.06153,,0,off,,,0,,0,,,,,,0,,0,,,,,0,,,PinwheelRefresh18,1,,,2023.32.7 bb3bd80f1e,0,,68,,51,,1069,0,0,0,,43,0,0,269.77,,,0,,,0,0,0,0,0,

That is line 7979 which from what I can tell is the first line that errors out