SparkFun Forums 

Where electronics enthusiasts find answers.

Have questions about a SparkFun product or board? This is the place to be.
By cpix
#100756
Hi all!

- I am having a weird problem with the OpenLog. When connected directly to a hardware-based serial (for example RX/TX-Pins 0/1 at an Arduino) I can easily use the logger.

However, when I connect the logger instead to some other digital pins, say Arduino pins 3&4, and use the NewSoftSerial-library, I get transmission errors (I issued a "?" after the prompt, later a "ls" to test transmission):
>?
OpenLog 1.61
Available cmmands:
new <file> : Creates <filAf sod6st e et:e
eldn cnoSfo ie etfrhonite
ioSryeiemEdo f tt
>

>ls
CONFIG.txt 12
>
Clearly, not all characters are transmitted properly. Seems to start out well and to fail later in the sequence... .

The code I am using on the Arduino is very simple:
Code: Select all
#include <NewSoftSerial.h>

NewSoftSerial mySerial(2, 3);

int state = 0;

void toggle_LED()
{
  if( state==0 )
  {
    digitalWrite(13,HIGH);
    state = 1;
  }
  else
  {
    digitalWrite(13,LOW);
    state = 0;
  }
}

void setup() 
{
  Serial.begin(57600);
  mySerial.begin(9600);
  pinMode(13,OUTPUT);
}

void loop() 
{
  if (mySerial.available()) 
  {
    char c = mySerial.read();
    if( c=='\r') 
      Serial.println( c,BYTE );
    else
      Serial.print( c,BYTE );      
  }
  if (Serial.available()) 
  {
    toggle_LED();
    char c = Serial.read();
    if( c=='\r') 
      mySerial.println( c,BYTE );
    else
      mySerial.print( c,BYTE );      
       
  }  
}
- am I missing here something in my program? Or is the problem here caused by speed mismatches between the Arduino and OpenLog?

Any help would be appreciated!

cpix
User avatar
By sparky
#100906
Hrmm - sounds like a buffer/timing problem. You can't really read in serial via software and then output it via hardware. You're bound to lose some characters.

I'll try to do some testing next week to make sure it's not an OpenLog problem, but my guess is that you are missing software serial characters coming in because Arduino is busy doing other things (like overhead of Serial.print or mySerial.print). One option would be to try to buffer all characters coming in. Here's untested pseudo-code:
Code: Select all
#define BUFF_SIZE 128
char i = 0;
char buffer[BUFF_SIZE];
while (mySerial.available())
{
  buffer[i++] = mySerial.read();
  if(i == BUFF_SIZE) break; //Bail! The buffer is about to explode
}
for(char j = 0 ; j < i ; j++)
{
  if(buffer[j]=='\r')
    Serial.println(buffer[j], BYTE);
  else
    Serial.print(buffer[j], BYTE);     
}
Again, untested, but I hope you can see where I'm going with this...
By cpix
#101074
- thanks very much for the hints. I modified the code to use your suggested buffered approach, but the results were inconclusive. Sometimes, I did get some correct response from OpenLog to commands, but again only partially, sometimes, the whole string send back was garbled.

I investigated the issue a little bit further however.

As described, I used initially the above (unbuffered) code in the following setup, which simply pipes commands to the OpenLog device coming in from the UART of the Arduino like this:
PC <-> COM2 <-> Arduino UART <-> NewSoftSerial(2,3) <-> OpenLog
This resulted in the described transmission errors. Note that commands to the OpenLog-device are routed in this setup over a softserial link on pins 2/3, and answers of the device back the chain accordingly. Again, this setup does not work, neither in buffered nor in unbuffered modes.

In a further experiment I "switched" the "device" talking to Openlog, by using the following different setup
PC <-> COM2 <-> FTDI Basic <-> NewSoftSerial (2/3) on Arduino <-> Arduino UART <-> OpenLog
with the same command-mirroring program as above. Here, the Arduino UART is talking to OpenLog (instead of NewSoftSerial). Otherwise, the setup is pretty much the same as the original one.

And: - this setup works! (PC to Arduino speed was on 57600, OpenLog on default speed 9600 in both cases, but I am rather sure that the speeds do not matter much).

It seems to me that I have found the following cases:
  • a) a connection to OpenLog via the Arduino hardware UART works as expected
    b) a connection to OpenLog via a NewSoftSerial interface does not work
So I think it is indeed a timing issue what is causing the problem here. It seems that somehow the OpenLog device has a timing which is not handled well with the NewSoftSerial lib (or the other way around).

Of course, my whole starting point was to connect the OpenLog to some digital pins other than the RX/TX hardware pins of the Arduino. For one thing, you have to unplug the OpenLog otherwise each time you reprogram the Arduino. I also want to use the UART for other communication purposes in my projects which is not possible if the OpenLog device requires to be connected to a hardware UART....

Btw: the issue seems to be reproduecable: a second OpenLog device I tested showed the same characteristics.

Is there any chance to get this device going with the NewSoftSerial or a similar library in the way described? That would be most helpful!

Cheers, cpix
By cpix
#105050
it's been a while, but: ... coming back to the issue... :wink:

I found this old code of Nathan Seidle (viewtopic.php?t=21438) where he's also using NewSoftserial to communicate with OpenLog.

When I run this code, I get (predictably) not the results expected:
Code: Select all
Deleting file: test.log
Creating file: test.log
Weird1: 

Weird1: 

Weird1: 

Weird1: 

Found it!!: 99
Weird1: Y
Weird1: a
Weird1: y
Weird1: 

Weird1: 

Weird1: >

The above output was copied right from the Arduino IDE serial monitor.

I did this test with two different OpenLogs units, with different SD-cards, with different Arduino-board - always with the same issues.

These results were obtained with a simple setup using a standard Duemilanove and an OpenLog, connected by 4 wires: Vcc was connected to 5 Volts, GND to GND, digital pin 8 to TXO, digital pin 9 to RXI.

Obviously, what OpenLog is doing is not what it is indented to do.

Checking the SD reveals that the file "test.log" gets created, but has some funny chars in it. I am currently using the lastest NewSoftserial-lib, but the problem did show up with earlier versions as well. Since that stuff works fine when using the hardware UART, it looks like OpenLog is quite picky about baud rates?

Does anyone has a clue about what is going on?

- cpix
By coyote20000
#105081
RX using software can be glitchy. The interrupts have to catch the data at the right time or it's lost. TX using newsoftserial should work perfectly fine.

If I were you, I'd issue a command using newsoftserial and ignore the response. Basically issue commands and then wait a bit before the next command. If you need to read the data back then you need a buffered serial line. Either put the openlog on the hardware side or get an Arduino mega with 4 real ports.

Maybe issue a command wait for a known 'command successful' response. On a no 'correct response' timeout, re-request the command. Loop.... I don't know if that would work, but it might be worth a try. Perhaps slowing the baud rate so more interrupts occur.

Dave
By cpix
#105086
- thnx Dave for your comments! I fully agree with your analysis.

However, I have already tried to lower the transmission speed, and the above example from Nathan Seidle is in fact running @9600. (I tried even the whole range of supported speed with my own software, but with the same results.)

One interesting point you mentioned is that TX should succeed in any case, as the OpenLog is using the more tolerant hardware UART. I had overlooked that fact until now!

However, the test with Nathan Seidle's code cited above creates a "test.log" file, and I have checked after reading your reply the content of the file created: weird chars are found in the "test.log" file - suggesting that even TX does not succeed fully.

Well, I might need to get access to a digital scope to do some timing analysis, but at the moment the thing seems to boil down for me to the following:

* Serial communication with OpenLog using hardware UART works.
* Serial communication with OpenLog using NewSoftserial does not work.

This was tested with the 2 OpenLog devices (running 1.5 and 1.6 versions of the OpenLog code) and various Arduino boards.

A further point worth noting: if I replace the OpenLog device with another Arduino (or Boarduino, or Arduino Pro Mini), the serial communication link using NewSoftserial works like a charm!

Therefore, at the moment I strongly suspect a problem in the serial communication part of the OpenLog device(s).

... and yes, indeed, a Arduino Mega should solve the problem ;)

- cpix.
By coyote20000
#105436
Sorry it took a while, I had a dead microSd chip. :(

Here's what I found so far.
When reading data using nss, you have to force the arduino to wait till all the characters are done being received.

Instead if using "if (nss.available()){}" use "while(nss.available())". This way nss isn't released and data isn't lost. Using "if" will read one byte then drop out. "while" will keep the arduino working on the nss stream till it's empty. If a byte comes in when nss isn't monitoring, it is lost...

As for sending data to OpenLog, I haven't written any code to test it, but I do know that OpenLog dumps it's buffer after a certain amount of data is collected. I believe I remember reading that when it dumps, it may lose data if the steam is continuous.

Post your nss/OpenLog test code and ill give it try...

Dave
By cpix
#105451
Hi Dave,

thanks for your comments! The interesting thing is that even the original code by Nathan Seidle (link above) fails with my 2 OpenLogs units, in both transmit directions.

I simply might have two weird OpenLogs, as the same serial setup (using NSS), only between two Arduinos, works flawlessly.

In future projects, I will be using the hardware UART of the Arduino for communication with the Openlog device and NSS for the communication link to the outside. That should work.

Thanks for your help! - cpix
By Eule
#129005
Hi
I can confirm the issue in the first post. I have tested 2 openLog devices on an EtherTen (an Uno type clone) using the NewSoftSerial library. I am using the above suggested
Code: Select all
while(nss.available())
commands for receiving return data.

The ? will produce the following result:
Code: Select all

OpenLog v2.5
Basic commands:
new <file>		: Creates <file>
append <file>		: Appends text to end of <file>. The text is read from the UART in a stream and is not echoed. Finish by sending Ctrl+z (ASCII 26)
md <directory>	: Creates a directory called <directory>
ls			 Shos th cotentof he crren dirctor. Ue widcar to o a ildard istig o fies i curent irecory
rea <fie> <tart <lngth <tye>		 WrtesASCI <legth>pars of<fil> tothe ermnal tartng a <strt. Omt <sart>and lenth> o red whle ile.<typ> 1 rint inASCI, 2in HX.
size<fil>		:Writ sie offileto trminl
isk		: howscar manfactrer,stats, ilesstemcapaity nd ree torae sace
rest			 Caues uit o reet ad usng ay nw paametrs i cofigfile


Mnus:
set		:Menuto cnfigre yste mod
bad			 Mnu t conigur bau rae
>
I have rx set to pin 6 and tx set to pin 7. GRN connected to reset on the Arduino. Otherwise I have GND and VCC to 5V connected. Changing the baud rate does not improve things.

Is there any advice on how to fix this?

Cheers