Despite the fact that we’re deeply involved in the world of real-time data, we have to accept that there are times when data needs to be captured at a specific point and then saved to a file. And that’s how our CSV Logger product came into being.
The way this works is that it watches a whole load of records (either from a specified list or the entire contents of a dataset) and maintains an up-to-date cache of their latest values. Then, at various pre-specified times during the day, it saves the whole lot into a CSV file for offline analysis, along with a header row identifying which field is which. It does loads of other clever stuff, like mapping multiple input fields onto a single output (just in case your data isn’t 100% consistent), as well as giving you the opportunity to tag each row with more information about the record.