Data from CSI loggers

LI-COR Environmental Forum » Ecosystem Exchange (Eddy Covariance) » EddyPro

(5 posts) (3 voices)

Started 1 year ago by TimL
Latest reply from TimL

Tags:

TimL
Member

Hi,

I'm from the Australian Ozflux community. I recently read A Brief Practical Guide to
Eddy Covariance Flux Measurements and then followed on with the 5 part series in your webinar (I wish I'd been led to it all sooner!).

At Ozflux, we have our own processing pathway. The raw fluxes are process in the Campbell Sci loggers (but we keep both the raw data and 30min averages) and then analysed through a Python coded pathway.

I'm interested in how EddyPro compares and also in describing our footprint from the data itself (currently, I use the rule of thumb for my 20m tower and analysis of wind direction). What I've found with EddyPro is that the raw data is broken up into files relating to the averaging period. Our raw data on the other hand is broken up only by collection periods (ie. in my case, roughly once a month).

Is there a way of tell EddyPro that it's a long file that needs to be averaged into 30min points or will it only work to raw data file lengths for its averages?

Posted 1 year ago #
Tom
Member

Howdy,

seems like the max file duration in Eddy Pro is 999 min, but if you are working with CSI loggers, you might have access to the LoggerNet software. Usually included is a tool called "Card Convert" which allows you to split long raw time series files into smaller (e.g. 60 or 30 min) ASCII or binary files. Raw file duration and averaging period don't have to be identical.

Best regards

Tom

Posted 1 year ago #
TimL
Member

Hi Tom,

I hadn't ever looked into the options of card converter - I always simply converted the full data content into a raw file. Great to know I can quickly split the data (I had looked into the LoggerNet file splitter, but it wasn't a straight forward process by any means).

Thanks for your help!
Tim

Posted 1 year ago #
gerardo
Member

Hi Tom and Tim,
sorry for addressing your question so late.
As you figured out already, EddyPro can only process "short" files. More precisely, the maximum length of each raw file is 24 hours if collected at 20Hz, or 48 hours @10Hz. As Tom explained, the LoggerNet program contains a tool for splitting long data files collected with CSI loggers into chunks of a specified length. We believe that it is best practice to collect a larger number of short length rather than a few huge files. Besides that, the length of the admitted raw files is limited in EddyPro by the need to keep memory allocation within values that can be handled by any (also not-so-modern) computer.
Regards,
Gerardo

Posted 1 year ago #
TimL
Member

Thank you Tom and Gerardo!

My problem turned out to be that I had already converted the TOB1 files into long TOA5 files through file converter, making splitting the files difficult and time consuming.

I went back to the LoggerNet manual and found that they include a small toa_to_tob1 program. In converting the files back into binary, the process of converting the files into 30min files was easy.

My problem now has been that when I try to work with a large number of 30min files, I get a persistent error, addressed by Gerardo already in the FAQ. What I've done instead is place the 30mins into folders for each month and process each separately.

I know that it should work for as much as a year's worth, by I've found the same error occurring when I selectively try to process as little as four months or so.

It's a little slower processing it this way, but not significantly and with it working in the background, I hardly notice it. From my first couple months of data, removing the flux data with a flag of "2" has yielded some interesting results which I'm planning to check alongside our in-house processing. I also like the fact that your process provides details on the actual flux footprint (we make a general assumption, based on height etc).

Thanks for all your help!

Posted 1 year ago #

RSS feed for this topic

Reply

You must log in to post.