Page 1 of 1

Unable to post data regularly

Posted: Wed Dec 16, 2015 12:29 pm
by clipcorner
Are there problems with the data server?
After successfully posting data every 15 minutes since November 14 (2015), since just after 19:15Z on 2015-12-15, my data is posting only intermittently & irregularly. Rarely every 15 minutes; typically gaps of 45 minutes up to 2 hours. Unfortunately I am remote from the device (hence its utility) so I'm not 100% certain it is sending data every 15 minutes. But in the month of close monitoring it never skipped a report, and I am able to verify that the remote wi-fi is functioning.

Re: Unable to post data regularly

Posted: Thu Dec 24, 2015 8:27 am
by Miskatonic
Is this still an issue? We have occasional server drops during maintenance, but if there is a large scale rolling outage, we know about it almost immediately from users.
Is it possible your power supply is on the fritz? Sometimes when battery/solar powered projects get to a low current star this appears, particularly with RF applications in the loop.

Re: Unable to post data regularly

Posted: Mon Jan 04, 2016 4:29 pm
by foxt
This is still an issue at the moment. HTTPS posts are not being processed from my electric imp agent, but HTTP posts are. Over on the electric imp forum, Hugo has done a bit of investigating and suggested that the problem is on the sparkfun side. I know that others have reported the same problem with HTTPS posts from their imp agents. The problem seems to have started several days ago, when HTTPS posts were failing intermittently. Today, they fail 100% of the time (at least for me). Again, HTTP posts succeed, perhaps that is a clue?

Re: Unable to post data regularly

Posted: Wed Jan 06, 2016 9:58 am
by agilb
some of my streams have dropped and aren't posting any longer.

Re: Unable to post data regularly

Posted: Sat Jan 09, 2016 10:36 pm
by Minimizer
Good to read I'm not alone here or missing something .. my posts also (from an Electric Imp agent) stopped abruptly on or about Dec. 30. Until then I'd been successfully posting using the basic SF code here ..

https://learn.sparkfun.com/tutorials/pu ... ectric-imp

Re: Unable to post data regularly

Posted: Sun Jan 10, 2016 10:23 pm
by hfiennes
As @minimizer and @foxt have noted, some imp users are seeing issues posting with https (and only https) to data.sparkfun.com. Moving to http avoids the issue, though is obviously not ideal.

I took some tcpdumps from the imp server end and I can often see the sparkfun end disconnecting immediately after the TLS client hello is sent. The issues are a lot more frequent every 5 mins or so, for a few seconds near the top of the minute, eg:

2016-01-01T04:15:03-05
2016-01-01T04:20:01-04
2016-01-01T04:25:01-03
2016-01-01T04:30:02-06
2016-01-01T04:35:01-06

...which to me implies that lots of people are posting with https at aligned times and the server is getting a bit overloaded/upset just for a few seconds.

I can provide more info if needed, our customers are doing many many many thousands of requests to data.sparkfun.com per day :)

Re: Unable to post data regularly

Posted: Tue Jan 12, 2016 4:13 pm
by foxt
Thanks for the insight. I switched to http requests to get around the problem, but I would like to switch back. I'll look at programming the imp to try to connect at different times and see if that helps. In the meantime, this had been working well for months and just started having an issue late December. Do we know if anything changed on the sparkfun side recently?

Re: Unable to post data regularly

Posted: Wed Jan 13, 2016 5:21 pm
by Miskatonic
We are looking at the IT side of things. There has been some DOS activity on Phant lately that has caused some issues. Also we are seeing users "over throttle " their feeds. If you are over the 100post/15mins, your feed will shutdown for an hour. This is how we keep bandwidth up for all users. The particulars of this can be read here: http://phant.io/docs/input/limit/

Re: Unable to post data regularly

Posted: Thu Jan 14, 2016 3:26 pm
by foxt
Thanks for the response. My imp project posts to sparkfun via the imp agent service every 15 minutes. Do my posts appear to be coming from imp, or from me? I suspect they look like they are coming from imp, and when thinking about the great tutorials etc. that show how to make the imp work with sparkfun, I bet that there are many many people like me. Does that make the imp service look like one of those users who "over throttle"?

Re: Unable to post data regularly

Posted: Thu Feb 04, 2016 12:42 am
by hfiennes
I just looked in our request logs and things appear to be working much better now. I've not gone back to see when things started working again, but I'm seeing successful connects with HTTPS consistently now. Thanks!