Page 1 of 1

Posting data to SparkFun server

Posted: Wed Apr 30, 2014 8:18 am
by sparky
Hi all,

I wanted to give the forum readers a heads up about a data channel service that we've been working on. Check it out here: http://data.sparkfun.com.

I was thrilled by how easy Wunderground was to work with and then let down by the complexities of using other services. So we created Phant (as in elephants don't forget) to make it super easy for people to push data out to the cloud.

For example, if you have a remote float sensor monitoring the water level of a de-watering pit (as we do in the new SparkFun building) and you want to report the water level every few minutes, where do you push the data? I'm a hardware guy. I don't know how to setup a server to save my life, so data.sparkfun.com fills the gap. All you do is create a stream, record the keys, then have your remote sensor concatenate a string.

http://data.sparkfun.com/input/Jxyjr7Dm ... e%20forum!
(go ahead, you know you want to click it!)

You can see the stream ID [Jxyjr7DmxwTD5dG1D1Kv] and the private key [gzgnB4VazkIg7GN1g1qA]. I don't know how to maintain a SQL database but I know how to get my Arduino to stick some strings together and then post it to HTTP (via cellular, wifi, or ethernet).

Then you can see the resulting data on the public stream: http://data.sparkfun.com/streams/Jxyjr7DmxwTD5dG1D1Kv

It's all free, open source goodness so if you don't trust us, need more than 50MB of space or faster than 10s update rates you can deploy Phant on your own server.

You're the first group to really hear about it outside the walls of SparkFun so we'd be thrilled if you kicked the tires and let us know what you think. We want to keep it simple (stupid) but what could be better?

Cheers,
-Nathan

Re: Posting data to SparkFun server

Posted: Wed Apr 30, 2014 8:25 am
by sparky
For a larger data set example, checkout the Wimp Weather Station tutorial and the feed it reports to. Here's the (relatively) simple code that pushes the data to the channel:
Code: Select all
    //Now we form the large string to pass to sparkfun
    local strSparkFun = "http://data.sparkfun.com/input/";
    local publicKey = "dZ4EVmE8yGCRGx5XRX1W";
    local privateKey = "private_key=myPrivateKey";

    bigString = strSparkFun;
    bigString += publicKey;
    bigString += "?" + privateKey;
    bigString += "&" + winddir;
    bigString += "&" + windspeedmph;
    bigString += "&" + humidity;
    bigString += "&" + tempf;
    bigString += "&" + rainin;
    bigString += "&" + baromin;
    
    //Push to SparkFun
    local request = http.get(bigString);
    local response = request.sendsync();

Re: Posting data to SparkFun server

Posted: Wed Apr 30, 2014 8:33 am
by phalanx
Hi Nate!

This is a great idea. Like you, my experience doesn't lie in the server side of the equation. I'll be curious to see the creative ways people utilize it.

-Bill

Re: Posting data to SparkFun server

Posted: Thu May 01, 2014 1:58 pm
by Kamiquasi
Tire-kicking has commenced - looks like it's okay with unicode, basic attempts at inserting html, and at least Bobby-proof.

I've also set up an actual data set - SparkFun Product Mutations nothing only one item in there yet, but should gradually fill out.

IRC answered some Q&A:
The timestamp column is always present - no need to add one yourself.
If the data exceeds 50MB, the oldest data starts getting deleted.

Re: Posting data to SparkFun server

Posted: Thu May 01, 2014 2:53 pm
by lyndon
This is awesome.

As luck would have it, I've been working on exactly the same thing in my spare time over the last week (well, a private version anyway). I'll be sure to check it out. Do you plan to support MQTT?

Re: Posting data to SparkFun server

Posted: Thu May 01, 2014 5:16 pm
by lyndon
And it looks like you've been visited by little Bobby Tables ;-)

Re: Posting data to SparkFun server

Posted: Thu May 01, 2014 5:39 pm
by Kamiquasi
oops.. it didn't like me submitting a value twice - e.g. &brewTemp=1&brewTemp=2 .. I killed your timestamp header :(

Re: Posting data to SparkFun server

Posted: Sat May 03, 2014 1:36 pm
by languer
Wow. Again, wow. This is so great and so welcomed. I had a wifly and ethernet shield, but I could not for the life of me make them talk to the outside world (outside of my router) without paying for a service.

Re: Posting data to SparkFun server

Posted: Sat May 03, 2014 9:03 pm
by Mee_n_Mac
I've been inspired by this to hack the "weather station" at the lake house. But instead of getting indoor and outdoor and lake water temps, it'll get outdoor and lake water temps and lake levels, posted in a publicly accessible way. I might even process the "fast" lake level data to give wave state and then add a anemometer ... data people going out to boat will find useful as it'll be timely (unlike the Weather Channel).

Something to do instead of drinki ... errr ... while drinking beer this summer !

Re: Posting data to SparkFun server

Posted: Mon May 05, 2014 6:17 am
by toddtreece
Kamiquasi wrote:oops.. it didn't like me submitting a value twice - e.g. &brewTemp=1&brewTemp=2 .. I killed your timestamp header :(
thanks for reporting the issue! it should now throw an error if you submit a field twice in the same request. i'll try to monitor this thread, but feel free to report bugs here: https://github.com/sparkfun/phant/issues

Re: Posting data to SparkFun server

Posted: Mon May 05, 2014 8:35 am
by toddtreece
lyndon wrote:This is awesome.

As luck would have it, I've been working on exactly the same thing in my spare time over the last week (well, a private version anyway). I'll be sure to check it out. Do you plan to support MQTT?
Thanks! MQTT support should be available in the next couple weeks.

If you would like to host your own private version of the server, check out: https://github.com/sparkfun/phant

Re: Posting data to SparkFun server

Posted: Wed Oct 22, 2014 5:56 am
by Tubes
Hey Sparky, this is all great stuff. Can you point me toward some examples on how to post data to data.sparkfun.com using a RedBoard and the Sparkfun SM5100B cellular shield? I am working on a project with these items and getting a head start would be great! Thanks.

Reuben

Re: Posting data to SparkFun server

Posted: Wed May 13, 2015 7:34 am
by agilb
I am curious and will start testing, but must all the fields be populated, or can one device update the wind speed while another updates the water level or temperature while making all the data available in the same log?

Re: Posting data to SparkFun server

Posted: Thu Dec 10, 2015 1:43 am
by nigelibrown
Here is some series data from a recent collection, if you look at the 4th line you can see the timestamp is incorrect i would expect it to be 12:12 this is happening quite frequently?

0 -69,20.67,69.22,2015-12-01T00:15:11.854Z
1 -72,20.68,69.21,2015-12-01T00:14:41.208Z
2 -72,20.68,69.27,2015-12-01T00:13:22.406Z
3 -71,20.7,69.19,2015-12-01T00:11:12.790Z
4 -71,20.66,69.2,2015-12-01T00:11:23.927Z
5 -70,20.72,69.28,2015-12-01T00:10:22.886Z
6 -70,20.68,69.23,2015-12-01T00:09:23.621Z
7 -71,20.69,69.23,2015-12-01T00:08:23.265Z