We are making a change to our interval data delivery to address situations where a utility updates readings for different units at different times. This may result in the same data intervals being sent more than once. Urjanet will implement this change on Tuesday, September 4.
Background: Currently, our system identifies the most recent interval for which we have extracted data for a unique combination of Provider, Account, and Meter (PAM). During extraction, we check our database to find this "latest interval end" of the most recent data for that PAM. Then, we look for any newer data on the provider's website after this most recent interval.
Issue: If a given PAM contains multiple units of measure, and values are only present through a given interval for a subset of those units, our system is currently using the maximum interval end for any unit as the latest interval end. When we next extract data for that PAM, we will look for data from this max interval end. This means we are not currently backfilling previous intervals with readings that were provided later by the utility.
Proposed change: Urjanet will include the Unit as part of a unique Provider-Account-Meter combination. If a given interval is missing data for any unit of measure, Urjanet will consider the prior interval to be the "latest interval end" for that PAM and look for updated data during our subsequent extraction. This will result in the same data interval being delivered more than once with updated data to fill in the remaining units of measure.
Our aim is to provide more complete, gap-free data, even when the utility website updates readings for different units of measure at different times.
Please reach out to your Urjanet Project Manager if you have any questions.