SparkFun Forums 

Where electronics enthusiasts find answers.

Everything pertaining to the data.sparkfun.com service, the phant project which powers it, and user projects which talk to the service.
User avatar
By sparky
#170666
Hi all,

I wanted to give the forum readers a heads up about a data channel service that we've been working on. Check it out here: http://data.sparkfun.com.

I was thrilled by how easy Wunderground was to work with and then let down by the complexities of using other services. So we created Phant (as in elephants don't forget) to make it super easy for people to push data out to the cloud.

For example, if you have a remote float sensor monitoring the water level of a de-watering pit (as we do in the new SparkFun building) and you want to report the water level every few minutes, where do you push the data? I'm a hardware guy. I don't know how to setup a server to save my life, so data.sparkfun.com fills the gap. All you do is create a stream, record the keys, then have your remote sensor concatenate a string.

http://data.sparkfun.com/input/Jxyjr7Dm ... e%20forum!
(go ahead, you know you want to click it!)

You can see the stream ID [Jxyjr7DmxwTD5dG1D1Kv] and the private key [gzgnB4VazkIg7GN1g1qA]. I don't know how to maintain a SQL database but I know how to get my Arduino to stick some strings together and then post it to HTTP (via cellular, wifi, or ethernet).

Then you can see the resulting data on the public stream: http://data.sparkfun.com/streams/Jxyjr7DmxwTD5dG1D1Kv

It's all free, open source goodness so if you don't trust us, need more than 50MB of space or faster than 10s update rates you can deploy Phant on your own server.

You're the first group to really hear about it outside the walls of SparkFun so we'd be thrilled if you kicked the tires and let us know what you think. We want to keep it simple (stupid) but what could be better?

Cheers,
-Nathan
User avatar
By sparky
#170667
For a larger data set example, checkout the Wimp Weather Station tutorial and the feed it reports to. Here's the (relatively) simple code that pushes the data to the channel:
Code: Select all
    //Now we form the large string to pass to sparkfun
    local strSparkFun = "http://data.sparkfun.com/input/";
    local publicKey = "dZ4EVmE8yGCRGx5XRX1W";
    local privateKey = "private_key=myPrivateKey";

    bigString = strSparkFun;
    bigString += publicKey;
    bigString += "?" + privateKey;
    bigString += "&" + winddir;
    bigString += "&" + windspeedmph;
    bigString += "&" + humidity;
    bigString += "&" + tempf;
    bigString += "&" + rainin;
    bigString += "&" + baromin;
    
    //Push to SparkFun
    local request = http.get(bigString);
    local response = request.sendsync();
User avatar
By phalanx
#170668
Hi Nate!

This is a great idea. Like you, my experience doesn't lie in the server side of the equation. I'll be curious to see the creative ways people utilize it.

-Bill
By Kamiquasi
#170674
Tire-kicking has commenced - looks like it's okay with unicode, basic attempts at inserting html, and at least Bobby-proof.

I've also set up an actual data set - SparkFun Product Mutations nothing only one item in there yet, but should gradually fill out.

IRC answered some Q&A:
The timestamp column is always present - no need to add one yourself.
If the data exceeds 50MB, the oldest data starts getting deleted.
By lyndon
#170704
This is awesome.

As luck would have it, I've been working on exactly the same thing in my spare time over the last week (well, a private version anyway). I'll be sure to check it out. Do you plan to support MQTT?
By languer
#170749
Wow. Again, wow. This is so great and so welcomed. I had a wifly and ethernet shield, but I could not for the life of me make them talk to the outside world (outside of my router) without paying for a service.
By Mee_n_Mac
#170757
I've been inspired by this to hack the "weather station" at the lake house. But instead of getting indoor and outdoor and lake water temps, it'll get outdoor and lake water temps and lake levels, posted in a publicly accessible way. I might even process the "fast" lake level data to give wave state and then add a anemometer ... data people going out to boat will find useful as it'll be timely (unlike the Weather Channel).

Something to do instead of drinki ... errr ... while drinking beer this summer !
User avatar
By toddtreece
#170787
Kamiquasi wrote:oops.. it didn't like me submitting a value twice - e.g. &brewTemp=1&brewTemp=2 .. I killed your timestamp header :(
thanks for reporting the issue! it should now throw an error if you submit a field twice in the same request. i'll try to monitor this thread, but feel free to report bugs here: https://github.com/sparkfun/phant/issues
User avatar
By toddtreece
#170790
lyndon wrote:This is awesome.

As luck would have it, I've been working on exactly the same thing in my spare time over the last week (well, a private version anyway). I'll be sure to check it out. Do you plan to support MQTT?
Thanks! MQTT support should be available in the next couple weeks.

If you would like to host your own private version of the server, check out: https://github.com/sparkfun/phant
By Tubes
#176181
Hey Sparky, this is all great stuff. Can you point me toward some examples on how to post data to data.sparkfun.com using a RedBoard and the Sparkfun SM5100B cellular shield? I am working on a project with these items and getting a head start would be great! Thanks.

Reuben
By agilb
#181938
I am curious and will start testing, but must all the fields be populated, or can one device update the wind speed while another updates the water level or temperature while making all the data available in the same log?
By nigelibrown
#186446
Here is some series data from a recent collection, if you look at the 4th line you can see the timestamp is incorrect i would expect it to be 12:12 this is happening quite frequently?

0 -69,20.67,69.22,2015-12-01T00:15:11.854Z
1 -72,20.68,69.21,2015-12-01T00:14:41.208Z
2 -72,20.68,69.27,2015-12-01T00:13:22.406Z
3 -71,20.7,69.19,2015-12-01T00:11:12.790Z
4 -71,20.66,69.2,2015-12-01T00:11:23.927Z
5 -70,20.72,69.28,2015-12-01T00:10:22.886Z
6 -70,20.68,69.23,2015-12-01T00:09:23.621Z
7 -71,20.69,69.23,2015-12-01T00:08:23.265Z