Missing observations from patrol data - errors when importing Json files

Hi everyone.
I have been importing patrols as Json files into my SMART system. During the importing process I keep on getting error messages for a lot of these patrols (not all) that certain features were not imported into SMART, and then upon closer inspection all the observations are missing from that patrol. Also, a lot of the end times of the patrols are incorrect - seemed to be auto filled to 23:59:59.

I have attached the list of features that were not imported into SMART from the patrol which is given during the importing process (the error). I have also included my log file from my system just after I have imported the files. I have the patrol file on hand if you need it but I cannot upload because of file size.
error log SMART desktop.txt (763 Bytes)
SMART error when importing Json files.txt (581.1 KB)

I do not have access to the field devices at the moment so I cannot generate any error logs from individual devices.

Any help would be appreciated.
Thank you.

Hi Alex,

I believe you’re importing via SMART Connect? Can I ask which version you are using?

We had issues of data processing order in 7.5.6, some of which were resolved in 7.5.7, so I just want to make sure you’re using the latest version of the software.

In this case the missing data is usually re-queued on the server and re-processing the data can solve the problem.

We sometimes also witnessed this issue when patrols are not properly ended on the Mobile device meaning the important “patrol end” cannot be found and imported into the database. Could this be possible?

Cheers,

Alex

Hi Alex.
Thanks for your reply.
No, the Json files are being manually imported into the SMART system because we are unable to use SMART connect where we work.

I am using 7.5.7 desktop, and smart mobile is the version I installed in beginning March 2024 (not sure if anything has been released after that).

Not a Connect issue in this case, but thanks for the extra information.

Thats interesting, but what do you mean not properly ended? THe patrol has to be ended in order for the data to be exported from the app onto the device/desktop. I also checked the Json file and there is a “SMART_ObservationType”: “StopPatrol” in the file, so I dont think it is that. I have however asked the field teams to make sure they are stationary when ending the patrol and waiting atleast 1 minute between their last observation and ending the patrol.

Any other insight into this problem? I am quite concerned because I cannot process this data and we are on a relatrively tight deadline.
Thanks.

Hi Alex,

Thanks for the additional info. If you’re using the SMART Mobile installation from the app store it should auto-update anyway.

Yes, we’ve seen issues in cases where the app is closed before an end point is generated, but it seems like this isn’t the case here.

Usually when this has occurred in the past, in which the order of files being processed is causing the blockage (i.e. the start point is not processed first etc.), re-processing usually solves it. The order of processing was an issue on Connect previously (now fixed) but I wasn’t aware of it being an issue with manual processing.

It’s possible the issue is occurring when sharing the files from the device.

I’ll escalate this issue. To confirm the process you’re taking is:

  1. Patrol teams are collecting data and exporting from device to internal storage
  2. Exported patrol data is shared from the devices’ internal storage (via a messaging platform?)
  3. You are then attempting to manually import this data (JSON format).

Thanks for clarification.

Cheers,

Alex

Hi Alex.
Good to know for the future.

Thanks for that, I will let the field teams know so I dont experience this going forward.

Thanks for all the extra information. The problem was a discrepency in my data model. In my data model I had a numeric attribute but because of the 3 decimal restriction, I had to make an additional text attribute with the same name. When I built the configurable model (which is what they are collecting from) I mistakeingly used the numeric attribute, which I have since deleted from my system because I did not want to use it by mistake again going forward. When I entered that that data this attribute was missing which resulted in a failure to import any of the observations and seems to have resulted in a mess of the patrol file (wrong times, missing station names, missing employee names). Now that I have readded this attribute and used the same key as before, it is working fine.

Sorry for wasting your time with this.
Thanks for all the help.
Cheers.

Hi Alex,

No problem. Thank you for clarifying the issue so others can avoid the same mistake.

Cheers,

Alex