Created on: 11 July 2015
An Arduino and Ethernet shield are used as an Arduino web server data logger that periodically logs data to a file on the SD card. The logged data can be viewed on a web page.
In the Arduino sketch for this project, the value from analog input A5 is logged to file together with the time in milliseconds from the millis() function. Any other data could be logged in the same way, such as temperature, pressure, voltage, etc.
Data can be logged at the interval set in the sketch. The example sketch logs data every 5000 milliseconds or 5 seconds.
An Arduino Uno, MEGA 2560 or similar Arduino board that is compatible with the Arduino Ethernet shield can be used in this project.
The following hardware is needed to build the Arduino web server data logger project:
The Arduino sketch code is based on the basic Arduino web server code. As shown below, the basic web server sketch has been modified to add logging capabilities.
After loading the web files to the micro SD card, insert it into the socket on the Ethernet shield. Load the sketch to the Arduino and use a web browser to surf to the IP address set in the sketch. The web page hosted by the Arduino will be displayed in the web browser. Click the link on the web page to load the log file to the web page, click the link periodically to refresh the log file on the web page.
This sketch implements the Arduino web server data logger and must be used in conjunction with the web files found below that are to be copied to the micro SD card.
Change the MAC address in the sketch to the MAC printed on the bottom of the Ethernet shield. Change the IP address to suit your own network.
// Basic Arduino Web Server version 0.1 modified to include logging // http://startingelectronics.org/software/arduino/web-server/basic-01/ // // More details and web files available from: // http://startingelectronics.org/software/arduino/web-server/01-log-data/ // // Date: 11 July 2015 // #include <SPI.h> #include <Ethernet.h> #include <SD.h> // maximum length of file name including path #define FILE_NAME_LEN 20 // HTTP request type #define HTTP_invalid 0 #define HTTP_GET 1 #define HTTP_POST 2 // file types #define FT_HTML 0 #define FT_ICON 1 #define FT_CSS 2 #define FT_JAVASCRIPT 3 #define FT_JPG 4 #define FT_PNG 5 #define FT_GIF 6 #define FT_TEXT 7 #define FT_INVALID 8 // pin used for Ethernet chip SPI chip select #define PIN_ETH_SPI 10 long log_time_ms = 5000; // how often to log data in milliseconds long prev_log_time = 0; // previous time log occurred // the media access control (ethernet hardware) address for the shield: const byte mac[] PROGMEM = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //the IP address for the shield: const byte ip[] = { 192, 168, 0, 20 }; // does not work if this is put into Flash // the router's gateway address: const byte gateway[] PROGMEM = { 192, 168, 0, 1 }; // the subnet: const byte subnet[] PROGMEM = { 255, 255, 255, 0 }; EthernetServer server(80); void setup() { // deselect Ethernet chip on SPI bus pinMode(PIN_ETH_SPI, OUTPUT); digitalWrite(PIN_ETH_SPI, HIGH); Serial.begin(115200); // for debugging if (!SD.begin(4)) { return; // SD card initialization failed } Ethernet.begin((uint8_t*)mac, ip, gateway, subnet); server.begin(); // start listening for clients } void loop() { // if an incoming client connects, there will be bytes available to read: EthernetClient client = server.available(); if (client) { while (client.connected()) { if (ServiceClient(&client)) { // received request from client and finished responding break; } } // while (client.connected()) delay(1); client.stop(); } // if (client) // log the data LogData(); }// void loop() void LogData(void) { unsigned long current_time; // current millisecond time File logFile; // file to log data to current_time = millis(); // get the current time in milliseconds if ((current_time - prev_log_time) > log_time_ms) { prev_log_time = current_time; // the log time has elapsed, so log another set of data logFile = SD.open("log.txt", FILE_WRITE); if (logFile) { logFile.print(current_time); logFile.print(" = "); logFile.println(analogRead(5)); // log the analog pin value logFile.close(); } } } bool ServiceClient(EthernetClient *client) { static boolean currentLineIsBlank = true; char cl_char; File webFile; // file name from request including path + 1 of null terminator char file_name[FILE_NAME_LEN + 1] = {0}; // requested file name char http_req_type = 0; char req_file_type = FT_INVALID; const char *file_types[] = {"text/html", "image/x-icon", "text/css", "application/javascript", "image/jpeg", "image/png", "image/gif", "text/plain"}; static char req_line_1[40] = {0}; // stores the first line of the HTTP request static unsigned char req_line_index = 0; static bool got_line_1 = false; if (client->available()) { // client data available to read cl_char = client->read(); if ((req_line_index < 39) && (got_line_1 == false)) { if ((cl_char != '\r') && (cl_char != '\n')) { req_line_1[req_line_index] = cl_char; req_line_index++; } else { got_line_1 = true; req_line_1[39] = 0; } } if ((cl_char == '\n') && currentLineIsBlank) { // get HTTP request type, file name and file extension type index http_req_type = GetRequestedHttpResource(req_line_1, file_name, &req_file_type); if (http_req_type == HTTP_GET) { // HTTP GET request if (req_file_type < FT_INVALID) { // valid file type webFile = SD.open(file_name); // open requested file if (webFile) { // send a standard http response header client->println(F("HTTP/1.1 200 OK")); client->print(F("Content-Type: ")); client->println(file_types[req_file_type]); client->println(F("Connection: close")); client->println(); // send web page while(webFile.available()) { int num_bytes_read; char byte_buffer[64]; // get bytes from requested file num_bytes_read = webFile.read(byte_buffer, 64); // send the file bytes to the client client->write(byte_buffer, num_bytes_read); } webFile.close(); } else { // failed to open file } } else { // invalid file type } } else if (http_req_type == HTTP_POST) { // a POST HTTP request was received } else { // unsupported HTTP request received } req_line_1[0] = 0; req_line_index = 0; got_line_1 = false; // finished sending response and web page return 1; } if (cl_char == '\n') { currentLineIsBlank = true; } else if (cl_char != '\r') { currentLineIsBlank = false; } } // if (client.available()) return 0; } // extract file name from first line of HTTP request char GetRequestedHttpResource(char *req_line, char *file_name, char *file_type) { char request_type = HTTP_invalid; // 1 = GET, 2 = POST. 0 = invalid char *str_token; *file_type = FT_INVALID; str_token = strtok(req_line, " "); // get the request type if (strcmp(str_token, "GET") == 0) { request_type = HTTP_GET; str_token = strtok(NULL, " "); // get the file name if (strcmp(str_token, "/") == 0) { strcpy(file_name, "index.htm"); *file_type = FT_HTML; } else if (strlen(str_token) <= FILE_NAME_LEN) { // file name is within allowed length strcpy(file_name, str_token); // get the file extension str_token = strtok(str_token, "."); str_token = strtok(NULL, "."); if (strcmp(str_token, "htm") == 0) {*file_type = 0;} else if (strcmp(str_token, "ico") == 0) {*file_type = 1;} else if (strcmp(str_token, "css") == 0) {*file_type = 2;} else if (strcmp(str_token, "js") == 0) {*file_type = 3;} else if (strcmp(str_token, "jpg") == 0) {*file_type = 4;} else if (strcmp(str_token, "png") == 0) {*file_type = 5;} else if (strcmp(str_token, "gif") == 0) {*file_type = 6;} else if (strcmp(str_token, "txt") == 0) {*file_type = 7;} else {*file_type = 8;} } else { // file name too long } } else if (strcmp(str_token, "POST") == 0) { request_type = HTTP_POST; } return request_type; }
The following files must be copied to the micro SD card, although the favicon.ico and robots.txt are optional.
This is the main web page displayed when a web browser connects to the Arduino web server.
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Arduino Web Server Logging</title> <link rel="stylesheet" href="styles.css"> <script src="script.js"></script> </head> <body> <h1>Arduino Web Server Logging</h1> <p><a href="log.txt">Fetch log file</a> to refresh log data.</p> <p>Log values:</p> <iframe src="default.htm" name="icontent" id="icontent"></iframe> </body> </html>
This web page is the default page displayed in the iframe of the index.htm page before the logged data is displayed.
<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Load Log File</title> </head> <body> <p>Click link to load log file.</p> </body> </html>
A simple style sheet straight from the basic web server version 0.1 web files. It only changes the background colour of the index.htm web page, but can be used to add custom styles to the project.
body { background-color: #DFEBED; }
This script is used to load the log file to the iframe window when the link on the page is clicked.
window.onload = initLinks; function initLinks() { for (var i=0; i<document.links.length; i++) { document.links[i].target = "icontent"; } }
Used to keep web spiders / bots from indexing the web page if the Arduino web server is put on the Internet instead of on a local network.
User-agent: * Disallow: /
Use your own favicon.ico file, or copy the Arduino favicon file (right click and choose Save Link As or similar menu item, depending on browser).
The web server part of the code can be understood by studying the code from the basic web server version 0.1. Web server principles and technology is explained in the Arduino web server tutorial series.
The basic web server code was modified to add the LogData() which logs the analog data to a file on the SD card.
A variable near the top of the web server sketch can be changed to set the interval period that data is logged in milliseconds:
long log_time_ms = 5000; // how often to log data in milliseconds
In the sketch, data is sampled every 5 seconds, or 5000 milliseconds.
Calling the LogData() function periodically in the main loop of the sketch, ensures that the time between each logging event is checked. If the time period since the last log has expired and the log file can be opened, the analog input A5 is read and saved to the log file together with the current millisecond time.
LogData can be modified to log the desired values, whether they are additional analog values or data from a sensor.
After at least one value has been logged, the link in the Arduino hosted web page can be clicked to display the contents of the log file.
The sketch uses a fairly long delay between logging data and is intended for logging values at longer intervals, preferably minutes apart.
Long intervals are necessary for logging as logging can not take place while the Arduino is busy serving up a web page to a browser. The bigger the web page, the longer it will take to serve up the web page and the longer the period will be that data can not be logged.
If faster or more accurate logging periods are needed, it will be necessary to use interrupts. Some buffering would need to be used in conjunction with the interrupt so that the SD card is not written to in the interrupt service routine (ISR).
For projects that are logging data such as temperature throughout the day and only need sample rates of 5 or 10 minutes, the example sketch should be sufficient.
The web server is based on the basic web server code.
Calculating the sample intervals using the millis() function was taken from the BlinkWithoutDelay example sketch.
Logging analog values to an SD card file was taken from the Datalogger example sketch.