I’ve had a lot of packages fail to process from Connect across a number of patrols and have been trying to figure out why. I suspect there’a a fair bit of mobile user error possibly to do with patrols left running (even overnight) without being paused or stopped. But in at least one case there seems to have been a correct Pause and then Resume but subsequent packages for that patrol failed to process. Am I write to think that long time lags or large geographic shifts will cause packages to fail to process to the expected patrol?
What seems odd is that the track + position flags on the Connect alert map are showing a lot more track for one patrol than has processed into desktop. Lots of packages failed to process for this patrol (they are just waypoints). The track on the alert map makes me feel the data should be retrievable.
I can’t see how to direct a given json package to a specific patrol cos that would help a bit. Can this be done? The waypoint packages appear to only be linked to the patrol based on device ID and presumably date/time interval and location?
Many thanks
Jeremy
Hi Jeremy,
Could you please capture a bug report from the device and send it my way? This is off the main Settings page at the bottom.
Thanks!
The device is remote from me but I certainly will send it if/when I can get hold of the file. Thanks
Justin I’m noticing that the processing failures all relate to track point packages. I guess this is because these packages don’t contain the patrolID (any reason why they couldn’t?) so it’s harder to be sure which patrol they relate to. The problem starts when a track point package has a significant time jump form the latest one that was processed (e.g. it jumps to the following day). I can see in some cases a time jump happening within the same track point package. I’m sure this is due to improper use of the app - i.e. failure to end a patrol causing the app to gather further track locations. But I do just wonder whether poor network connection, the order in which packages are assembled and uploaded (they aren’t necessarily in time order I notice), or some other technical issue could create the same scenario (e.g. a pause patrol message goes missing)
In case anyone’s reading this!.. I’ve resolved all the processing failures by editing the times associated with each patrol date in SMART desktop. The problems arose because users left their patrols running, sometimes for days. If they made an observation or did an operation like pause/resume then SMART desktop could tell which patrol it belonged to because those data packages carry the patrolID. And then Desktop would make sure there was an appropriate date associated with that patrol (and all dates in between from when the patrol started), but the default time given for that date would just be from the time of that observation onwards, until 23:59:59. Any track point (matching DeviceID only, no PatrolID to match) with a time outside of this time interval would fail to process. So by editing the start time for the patrol date to 00:00:00, and even the end time to 23:59:59, and rerunning the connect processing, allowed all the stuck packages to process successfully. I could then update start and end times from the track point data.
If you have a bug report from any of the devices, that would be great. There was an issue where if you press “back” when editing a sighting (instead of confirm or cancel), then the app would crash. That’s been fixed in the meantime, but we are still trying to understand the conditions needed for an empty patrol id.
Thanks Justin. Jist to be clear I’ve not noticed any missing Patrol Id for observations or patrol start /end events. I was commenting that the track points don’t include PatrolID. This seems to be by design but I wonder why it needs to be this way.
Hi Jeremy,
I am having a similar issue. It looks like most, if not all, of the processing errors I am encountering have to do with waypoints.
Although, I am not sure if the issue I am having is a result of times. In any case, maybe I can give it a try. How did you edit the start and end times? Did you do it through a JSON editor application?
Thank you so much for your time!
Best,
Maegan
Hi Maegan,
I edited the start and end times in SMART Desktop. I opened up the relevant patrol and changed the times that appear on the individual date pages for the patrol. By setting them to start at 00:00:00 and end at 23:59:59 for each date this ensured any waypoint packages (actually these are “track points” rather than waypoints) which matched the Device ID for that patrol and containing those dates, would find a home in that patrol+date combination. If the waypoint package had dates that didn’t appear in the patrol I extended the dates of the patrol in SMART Desktop (on the patrol’s summary page) to include the relevant date. (SMART automatically adds in all intervening dates but that doesn’t matter.) I then used the option on the patrol date pages to reset the start /end time using the waypoints and track points so that I once again had a reliable measure of patrol duration.
This does require you to have some idea which patrol the failing packages might belong to. The match is made using Device ID so you’ll want to know which patrol comes from which device. Device ID is probably recorded in SMART somewhere, I’m not sure where.
This for me has all been about recovering data from incorrect field usage so we are continuing with training to ensure all users do properly stop their patrols at the end of each day and don’t leave them running from day to day.
Hope it helps
Jeremy
Thank you for your reply. I am a bit new to SMART and SMART mobile so apologies if any of my comments/questions are a bit ill informed. Are you creating a blank patrol within the desktop app and then using SMART Mobile importer to add data to this created patrol (selected “add to existing patrol” and thent he patrol I created; picture attached) from JSON files saved from the SMART mobile app? I did it this way using your method of making sure the created patrol had 00:00:00 for the start time and 23:59:59 for the end time, but I still had 87 feature processing errors (out of 187 features) (picture attached). These errors are all for features with “observation type: waypoints”. The same errors occur when I use SMART Mobile importer to import a JSON file to a new patrol (“create new patrol”) as well.
I’m using Connect so the data packages are sent from the phones to the cloud and I use the process data queue function in desktop to download them. I didn’t need to create any patrols manually because the “Start Patrol” message was always present in the collection of files and that automatically created the patrol in desktop. So it seems a bit different to the issue you are having.
Ah. Gotcha. Thank you for talking this though with me!
I’m hoping to get a bug report from one of the phones and will share when I get it. I continue to have very high levels of failure with processing, from multiple users, under various conditions. I’ve been inclined to think this must be something to do with user error (e.g. patrol running over more than one day) but I now have examples where the user has done everything right as far as I can tell and it still fails. Latest example from today: a set of packages arrived on Connect and downloaded to desktop. These created 3 patrols in desktop. Five packages failed to download, of which one gave an ERROR the other four just requeued. I could see in desktop that two of the patrols had sensible looking end times and the third was 23:59:59 which implies to me that the STOP PATROL package hadn’t arrived. Sure enough when I looked at the requeued packages, the last one to be processed was indeed the STOP observation. I tried processing again but no improvement. I looked at the package that gave an ERROR. This contained two observations, one with two photos. All the other failed packages were single observations with 2 photos in each, all complete, with footer (which can sometimes be missing in packages with photos). I edited the ERROR package so that it only contained the first observation, uploaded, and this then processed OK. I also tried with the second observation and this worked too. The other four packages (including the stop message) still wont process. Sometimes I’ve found that deleting one photo in a package can help. But not always. And it is laborious to do. Other recent patrols had many failures of packages, often with observations, and stop messages. It’s impossible to know which patrol these failures are affecting if data have arrived from more than one patrol at once.
Hi Jeremy,
I am following up with the server side folks to get more information. This should be working. If it is possible to add me to your server, I can take a look more directly. Please send the details to justin@steventon.com.
The track points contain the deviceid, which is how the system knows which patrol they belong to.
Cheers,
-Justin
Thanks Justin. Added you and sent you email about it
I believe there was a bug fix for processing data through Connect, specifically related to patrols that cover more than 1 day, it was made in SMART 7.1 or 7.2 as I recall. Are you are using SMART 6 in this instance by any chance?
I’m using desktop version 7.4.0
If your data is not sensitive etc you could export and share your CA with us, that might help track things down. We are testing various scenarios etc and can’t seem to see any problems. Really just your data model and configurable model that you are using are the main things for us to review if you don’t want to share your patrol data etc.
I’ll message you my email address so you can share files if that works.
Also, if you are able to, create an admin user for us that we can use to access to your Connect server would also let us debug the issue more directly. Thanks!
Thanks for creating the user on your server for us, that helped quite a lot. We tracked down two separate issues, one of which is fixed now in smart 7.5.2. Once that is released (maybe a month or two before a public release, I’m not positive of the timeline yet) it should solve the problems. The other is on Smart Mobile’s side I believe, so I passed on the relevant info to Justin.