Welcome Guest [Log In] [Register]
Add Reply
Weather station software development
Topic Started: Dec 28 2009, 04:00 PM (1,361 Views)
Gina
Member Avatar

I'm starting a new thread for this as it is off-topic for the VirtualBox thread.

I have a plan for an automatic weather station application, written in Python and using GPL open-source libraries and modules. :) Of course, to implement something as all encompassing as Cumulus will require quite a number of modules but I've already found a lot of useful stuff. I also expect to incorporate webcam capture and display on the website. There's a webcam capture Python library. And I've found USB and ftp modules/libraries. A simple version should not take too long. Adding the various history data tables and charts will take longer.
Offline Profile Quote Post Goto Top
 
Phaedrus
Member Avatar

Guessing you`ve already looked at this, but if not might be useful. Maybe you could use some of it?
http://www.wviewweather.com/
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

No that wasn't one of the things I've looked at (but it is now :lol ) So thank you :) It doesn't support my particular flavour of AWS but might have some usable bits :)

In fact I have found some Python modules/libraries that do support my station series (Watson, Fine Offset /EasyWeather etc.) and have already tested the data acquisition side of things :) I have it reading data from my WS console and writing it to file. So that's one major step achieved :) :)
Edited by Gina, Dec 28 2009, 06:55 PM.
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

I'm concentrating on getting my AWS more accurate and reliable ATM while we have some decent weather. Applying some mods and resighting the temp/humidity and rain sensors. Just look at my rainfall figure :lol Isn't that amazing - two metres of rain in a few hours! :rofl Interference from electric fence - now fixed.
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

Well the interference may be fixed but I've tried everything I can think of to get rid of the now 3m of rainfall!! I've even removed the batteries in the WS (with USB disconnected) whcih has cleared the WS data memory. I've edited all the Cumulus data files and ini files. Left message on Cumulus forum.
Offline Profile Quote Post Goto Top
 
Phaedrus
Member Avatar

Sounds like a tricky challenge..
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

I've now tried absolutely everything I can think of :( I even removed ALL the batteries and USB connection. The tx batteries needed replacing anyway. So I put new batteries in the tx unit, replaced the batteries in the console, cleared out any stored data in the console memory, checked the rain data directly and it was zero, edited the various Cumulus files to remove remaining rainfall values. Then I double checked everything and could find NO daft rainfall data anywhere. So restarted Cumulus. It found 500mm odd of rainfall which it's added to today!!

I'm now at a total loss as to what to try next. At least as regards Cumulus. I'm getting a bit fed up with it - I'm spending hours on it and getting nowhere fast!! :( :(

Guess I might uninstall Easy Weather, wipe all it's date etc. and reinstall. Reinstall it and see what that says. I dunno... I did that with Cumulus and that still found false data and EW did in direct mode.
Edited by Gina, Jan 3 2010, 01:50 PM.
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

Some progress I think :) Uninstalled EW then deleted it's folder tree and reinstalled it and ran it. Viola!! No rainfall :) :)

Phew!!!

I've gone off Cumulus!! Editing the data to correct for something which is reasonably likely to affect a few people, like interference or simply moving the rain gauge, is exceedingly tedious. You need a text editor to edit a couple of comma separated text data files. That's every record during and after the interference in two places per record! Replace All can help with the data after the interference as long as there's no additional "rainfall". During the interference every record is different and needs two individual values edited per record.

A user function to clear rainfall (or set to a particular value) to correct erroneous data would not be difficult to write. TBH I think this is a major omission from Cumulus and I'm not the only one who thinks so - several other users have said this in the Cumulus forums.

I guess, in one sense, I'm wasting my time with Cumulus when I could be doing my own but it will be quite some time before it's ready for broadcasting the data and I want something running in the meantime.
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

Conquered it!!

More editing of Cumulus data and I've at last got rid of the faulty data :) Now I need to hibernate for three weeks! :lol
Offline Profile Quote Post Goto Top
 
Phaedrus
Member Avatar

Yay! :clap

Well done Gina. :thumbsup
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

AH well - we used to have a motto where I worked many years ago "The difficult we do straight away, the impossible takes a little longer" :rofl And I feel as if I've done the impossible :lol

Thank you Trev :)
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

Been looking at the Python scripts for my Linux AWS software again. Some progress :) I have data downloading from the WS, data processing with generation of text files and crude graphs. Been trying to get the ftp upload working. Got it running without producing errors and the router lights are showing data transfer but I can't find where it's going :lol Should be in weather/data.
Offline Profile Quote Post Goto Top
 
Phaedrus
Member Avatar

Sounds like a fair bit of progress. What are your plans for the gui?
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

Well, I've got the upload working manually - the files are now in the weather/data directory on the website - but when run from the full script it comes up with errors. Must have something a bit wrong in the Python script. Anyway, I think I'll do the web page for this system next. I presume this is what you mean by the gui - the local software is run off a cron job to get the data, process it and make graphs and then upload it to the website. It is headless ATM though I guess I'll add a GUI to it to set up calibration etc. and also to enter user data when I get to the stage of publishing it for others to use - but that's a way off yet :lol

I may have a website for public viewing later this evening if all goes well. Real live data may take a little longer :lol
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

I now have just the one web page that can be publicly viewed - http://ginad.org.uk/weathergd/Data-24-hours.html

Note this is just a first go - a very simple web page and very crude graphs.
Offline Profile Quote Post Goto Top
 
Phaedrus
Member Avatar

That is brilliant. To write your own application for your weather station like that. You should be very pleased with yourself :thumbsup

I guess its just tweaking the interface to get it to appear the way you like it now. I suppose really as long as the data is readable thats what counts, but personally I would want a really funky interface with dials a fancy graphics everywhere :cool
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

I'm afraid I can't take all the praise - most of the modules were already written, I just did some modifications. But I used to write my own software from scratch, later moving on to higher languages and library modules where some of the mundane stuff was already written.

Yes, I do intend to make very much nicer looking graphs and dials etc. There's a few other mods required too. To the way data is collected.
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

I've found out why the full script didn't work. I put debugging print statements at strategic places and narrowed it down until I found the culprit. A parameter missing. Put in the missing parameter and now I can run one cli instruction to get it to sync with the WS, grab the data, process it, draw graphs and upload the stuff to my website. There are still several "funnies" to sort out eg. something in there seems to have a love for 2008 but I don't have any 2008 data as I only bought the AWS last Feb and everything was cleared a couple of days into 2010.

I have made no attempt as yet to sort out a better graphing system or produce nicer web pages and images. But I've been adding more data to the website still using very crude pages. I want to get the data right first then I can tweek the web pages and look for better graph modules etc. Anyway, the latest results are available for viewing at http://ginad.org.uk/weathergd/home.html

Additions :- Page for 7 day data and wind data pie charts added to both 24hr and 7day pages. Added home page to access both data pages.
Edited by Gina, Jan 8 2010, 07:46 PM.
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

Improved the graphs a bit and added 28 day graph. Also added background image and changed home page. I have ideas for further improvements. The plot captions want moving outside the plot area - I'll have to do this separately as it's not possible within the plotting module. ATM I have all the plots on one page done in one graph call, so I'll have to make separate calls with a file for each plot and add the captions on the web page itself instead of in the plots. And the wind rose charts are a right mess! Got that to sort out too. That will mean mods to the windrose module itself.
Offline Profile Quote Post Goto Top
 
Gina
Member Avatar

Couldn't get my brain in gear enough for intricate coding so did some work on the graphs and web pages today. The website has now reached 6 pages so it's time to stop ad-hoc copying and editing to add pages and write a CSS file.
Offline Profile Quote Post Goto Top
 
1 user reading this topic (1 Guest and 0 Anonymous)
Go to Next Page
« Previous Topic · Computer Chat · Next Topic »
Add Reply