OpenLog large data handling

Have a good idea for a new product for SFE or Olimex? Let us know!

Moderator: phalanx

Post Reply
ijt
Posts: 2
Joined: Wed Nov 01, 2006 3:53 am

OpenLog large data handling

Post by ijt » Thu Jul 22, 2010 10:31 am

For one of my projects, we needed a little more data handling w/large data files (some more than 10mb). In main.c, just under the "read" <file> module, we added the following code for a simple handshake after each full text line:

else if(strncmp_P(command_arg, PSTR("rdlin"), 5) == 0) // ijt 7-22-2010
{
//Argument 2: File name
command_arg = get_cmd_arg(1);
if(command_arg == 0)
continue;

/* search file in current directory and open it */
struct fat_file_struct* fd = open_file_in_dir(fs, dd, command_arg);
if(!fd)
{
uart_puts_p(PSTR("error opening "));
uart_puts(command_arg);
uart_putc('\n');
continue;
}

uint32_t chunk_to_read = (uint32_t)-1;
/* print file contents, 1 line at a time. */
uint8_t buffer;
while((fat_read_file(fd, &buffer, 1) > 0) && (chunk_to_read > 0))
{
if( buffer >= ' ' && buffer < 127 )
uart_putc(buffer);
else if (buffer == '\n')
{
uart_putc(buffer);
if (uart_getc() == 0x1b)
break; // ESCAPE = cancel read
}

chunk_to_read--;
}
uart_putc(0x1A); // ctrl Z
uart_putc('\n');
fat_close_file(fd);
}

As you can see, it would be easy to add 'repeat' previous line, add chksum, etc.
Nice work on the main app Nathan!
Jack

User avatar
sparky
Site Admin
Posts: 1337
Joined: Mon May 05, 2003 6:02 pm
Location: Colorado
Contact:

Re: OpenLog large data handling

Post by sparky » Wed Aug 04, 2010 7:31 am

Hi Jack - thanks for the kind words! I'm always happy to work on OpenLog.

I'm trying to wrap my head around what your code does. It reads from a file, and prints it to the terminal, yes? How is this different from the current read command? We have start/stop arguments on the read command so you could do a :

read test.txt 5 173

Which would read from spot 5 to spot 173.

Lemme know. We're always adding more features (well, until we overflow the ATmega328 flash space...).

Cheers,
-Nathan
Pursue the cheese, win the excellence.

itai n
Posts: 16
Joined: Sat Nov 28, 2009 3:29 pm
Location: Haifa, Israel

Re: OpenLog large data handling

Post by itai n » Tue Aug 10, 2010 2:51 pm

Very nice work with OpenLog.

The upgrade to V2.x does not include the main.hex file, leaving a broken link in the product page. I only got 8GB card so I had to install and learn how to use the Arduino software (beeing "old school" I still prefer using Makefiles and avr-gcc).

An option that I would like to see in some future version of the product is the ability to log (binary data) using an I2C connection.

-- Itai

Post Reply