Trying to figure out an issue and can’t recall the answer (or if I’m overlooking something). A site we support processed data queue items from 24 Jan, but then shortly after, they did something to get their database out of sync with Connect.
They replaced the local CA with the databased from Connect, but since they couldn’t sync after processing the data, the patrols from the 24th are not in the database. I’ve tried to reprocess the items in the data queue but am getting the warning, “Could not process any of the data. Item has been requeued on server.”
Any ideas how to recuperate those patrols from the 24th?
Thanks!
Hey Drew,
If they didn’t successfully sync their changes back up to Connect after locally processing the data queue items from 24 Jan, it means that data were only on their local desktop and not yet in the Connect database. So, when they replaced the local CA from Connect, it wiped out those local changes. It may have been better to replace the Connect CA with the local desktop, since that’s where the most recent changes were.
Regardless, if you still have the completed data queue items from 24 Jan available in your Data Processing Queue it should be recoverable. Depending on what changes were made since the CA was refreshed from Connect, this may or may not be successful. At the very least, you could download and review those files from Connect and potentially reconcile the data to be able to process them successfully, though it may be a difficult process. It will likely require looking at the individual files to determine why they can’t be processed. Most likely, it’s due to multiple overlapping patrols from the same device on that date.
Thanks @matt.hron for the reply - very helpful. Is there any guidance on what that process looks like - downloading the individual files from the data queue and reviewing them? What you suggested was my first thought too (multiple overlapping patrols, potentially same device), but not sure what I should be looking for in the data queue file? Wondering if we have any guidance on that? Or potential indicators? Attaching some screenshots of dq files here. Appreciate any insights from you or others! Thanks!
Hi Drew,
Unfortunately, when you get to this point and there’s something that won’t process and not an obvious reason why, you need to take it on a case-by-case basis and there isn’t a standard repeatable process that I’m aware of to fix this. I’m no expert at this, but here are my thoughts.
-
The issue with data matching multiple patrols usually just applies to trackpoints, as those have far less data in them and can be ambiguous as to which patrol it applies to. All of the examples you shared here are observations (You can see in the SMART_ObservationType that one is a patrol pause and one is a stop patrol), and have plenty of metadata that tells you exactly which patrol they’re associated with, so I don’t think this is the case.
-
Other issues I’ve seen involve data being modified or processed out of order, such that the SMART_ObsCounter values get out of sync and don’t match up. All of the observations need to be processed in order, and the ObsCounter value needs to match the next expected observation number. This can be difficult to figure out the count correctly, but you can look to see if there’s anything obvious missing. Also, double-check that there are correct sequences of patrol start, pause, resume, and end.
I wish I had more insight, and I might be able to glean a bit more by comparing these files to the data already filed for the patrol in your SMART CA. If the data all looks as expected and we can’t see anything wrong, our next step would be to open a ticket for further troubleshooting.
