Arduino Inputs using Ajax with XML on the Arduino Web Server

Created on: 27 March 2013

Part 14 of the Arduino Ethernet Shield Web Server Tutorial

The Arduino web server hosts a web page (stored on the SD card) that displays the status of two push button switches and an analog (analogue) input.

The status of the two switches and the analog input are updated on the web page using Ajax. An XML file containing the switch statuses and the analog value is sent from the Arduino to the web browser.

This example produces the same output on the web page (with only the text changed) as part 9 of this tutorial – Analog Inputs and Switches using AJAX, with the following changes:

  • Switch statuses and an analog value are sent in an XML file and not as a block of HTML.
  • JavaScript responseXML is used instead of responseText to get the received values from the Arduino out of the XML file.
  • The values from the XML file (Arduino inputs) are inserted into HTML paragraphs in the web page instead of replacing the entire paragraph.
  • The web page is stored on the micro SD card of the Ethernet shield.

Why use Ajax with XML?

The advantage of using Ajax with an XML file is that individual values can easily be extracted by JavaScript on the web page, rather than having to write JavaScript code to extract values from a text file.

XML File Structure

An XML file uses tags like HTML or XHTML. The file has an initial tag that identifies it as an XML file. A main user defined tag follows that contains all other tags for the file.

This listing shows the structure of the XML file used in this example:

<?xml version = "1.0" ?>
<inputs>
    <button1></button1>
    <button2></button2>
    <analog1></analog1>
</inputs>

The inputs tag and all other tags contained in it are user defined names. The above XML could also be created as follows:

<?xml version = "1.0" ?>
<inputs>
    <button></button>
    <button></button>
    <analog></analog>
</inputs>

This file shows a button type and analog type that can be used to contain any button state or analog value. By adding more <button> or more <analog> tags, the state of additional buttons or analog inputs can be added.

The difference between the above two files is that the first uses unique names for all tags, whereas the second uses the tags to identify an input type.

Arduino XML File

In this example the Arduino creates an XML file and inserts the status of the switches and the analog input between the tags. The XML file is sent to the web browser in response to an Ajax request for data.

The image below shows an example of the XML file sent from the Arduino.

XML file sent by the Arduino
XML File Sent by Arduino

How Ajax with XML Works

If you have been following each part of this tutorial, then a lot of this will look familiar.

To update the Arduino input values on the web page, the following must occur:

1. Requesting a web page

As usual, the web browser is used to access the Arduino web server at the IP address that it has been set at in the Arduino sketch.

Connecting to the Arduino web server from a web browser
Connecting to the Arduino Web Server

This causes the web browser to sent an HTTP request:

GET / HTTP/1.1
Host: 10.0.0.20
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:19.0) Gecko/20100101 Firefox/19.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-ZA,en-GB;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive

2. Sending the web page

The Arduino web server receives the above request and responds with an HTTP header followed by the web page:

HTTP/1.1 200 OK
Content-Type: text/html
Connection: keep-alive

Arduino web page with JavaScript for Ajax

The Arduino reads the web page from the SD card and sends it to the web browser. After receiving the web page, it will be displayed in the web browser.

The web page contains JavaScript that is used as part of the Ajax process.

Note that the content type in the HTTP header for the HTML web page is text/html.

3. Ajax request

The JavaScript code on the web page sends an Ajax request to the Arduino (and continues to send a request every second).

GET /ajax_inputs&nocache=299105.2747379479 HTTP/1.1
Host: 10.0.0.20
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:19.0) Gecko/20100101 Firefox/19.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-ZA,en-GB;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip

4. The Arduino responds to the Ajax request

After receiving the request for the XML file, the Arduino responds with an HTTP header followed by the XML file which contains input values from the Arduino.

HTTP/1.1 200 OK
Content-Type: text/xml
Connection: keep-alive

XML file from Arduino

Note that the content type in the HTTP header is now text/xml.

5. Displaying the data

Finally the JavaScript in the web page extracts the three values from the Arduino from the XML file and displays them on the web page.

Arduino Sketch and Web Page

Web Page

The Arduino hosts the following web page on the SD card:

Arduino web page HTML and JavaScript code
Arduino Web Page index.htm - click for a bigger image

This is basically the same web page as sent by the Arduino in part 9 of this tutorial, but with the following changes (besides the text changes):

Function

The JavaScript function in the web page has been renamed to GetArduinoInputs().

The function still sends out an Ajax request every second. It now sends ajax_inputs with the GET request.

Because an XML file is being sent back from the Arduino, the function now checks if responseXML contains data instead of responseText:

if (this.responseXML != null) {

The data is extracted from the received XML as explained shortly.

HTML

The HTML is modified to display three paragraphs of text, one each for each value sent from the Arduino. Each paragraph contains an HTML span, each span has a unique ID.

The JavaScript function will insert the extracted values from the XML file into each span. This will replace only the default text (...) in each paragraph with the value from the Arduino.

The function uses the following code to get hold of each span for inserting data (code for getting "input1" shown here):

document.getElementById("input1").innerHTML =

Extracting the XML Data

The XML data is extracted from the received XML file using the following line of code:

this.responseXML.getElementsByTagName('button1')[0].childNodes[0].nodeValue;

In this code, this.responseXML is used instead of this.responseText as used in previous examples.

Now every tag in the XML can be accessed using this.responseXML.getElementsByTagName('button1') as can be seen in the JavaScript function.

If you refer back to the top of this part of the tutorial under XML File Structure, and the second XML file example, you will see that there can be tags with the same name. If we used this for the button tags, then each button tag value can be accessed as follows:

this.responseXML.getElementsByTagName('button')[0].childNodes[0].nodeValue;
this.responseXML.getElementsByTagName('button')[1].childNodes[0].nodeValue;

This is usefull if there were a number of buttons that you did not want to give unique tags to. The values can then also be accessed in the JavaScript by using a loop.

The button values will then be extracted in the order that they have been inserted into the file.

The number of buttons in the XML file can then be obtained by using:

this.responseXML.getElementsByTagName('button').length

Arduino Sketch

The Arduino sketch for this example is shown below.

/*--------------------------------------------------------------
  Program:      eth_websrv_SD_Ajax_XML

  Description:  Arduino web server that serves up a web page
                that displays the status of two switches and
                one analog input.
                The web page is stored on the SD card.
                The web page contains JavaScript code that uses
                Ajax and XML to get the states of the switches
                and value of the analog input.
  
  Hardware:     Arduino Uno and official Arduino Ethernet
                shield. Should work with other Arduinos and
                compatible Ethernet shields.
                2Gb micro SD card formatted FAT16.
                Push button switches interfaced to pin 7 and 8
                of the Arduino. Potentiometer interfaced to A2
                analog input.
                
  Software:     Developed using Arduino 1.0.5 software
                Should be compatible with Arduino 1.0 +
                SD card contains web page called index.htm
  
  References:   - WebServer example by David A. Mellis and 
                  modified by Tom Igoe
                - SD card examples by David A. Mellis and
                  Tom Igoe
                - Ethernet library documentation:
                  http://arduino.cc/en/Reference/Ethernet
                - SD Card library documentation:
                  http://arduino.cc/en/Reference/SD

  Date:         27 March 2013
  Modified:     17 June 2013
                - removed the use of the String class
 
  Author:       W.A. Smith, http://startingelectronics.org
--------------------------------------------------------------*/

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>

// size of buffer used to capture HTTP requests
#define REQ_BUF_SZ   50

// MAC address from Ethernet shield sticker under board
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192, 168, 0, 20); // IP address, may need to change depending on network
EthernetServer server(80);  // create a server at port 80
File webFile;               // the web page file on the SD card
char HTTP_req[REQ_BUF_SZ] = {0}; // buffered HTTP request stored as null terminated string
char req_index = 0;              // index into HTTP_req buffer

void setup()
{
    // disable Ethernet chip
    pinMode(10, OUTPUT);
    digitalWrite(10, HIGH);
    
    Serial.begin(9600);       // for debugging
    
    // initialize SD card
    Serial.println("Initializing SD card...");
    if (!SD.begin(4)) {
        Serial.println("ERROR - SD card initialization failed!");
        return;    // init failed
    }
    Serial.println("SUCCESS - SD card initialized.");
    // check for index.htm file
    if (!SD.exists("index.htm")) {
        Serial.println("ERROR - Can't find index.htm file!");
        return;  // can't find index file
    }
    Serial.println("SUCCESS - Found index.htm file.");
    pinMode(7, INPUT);        // switch is attached to Arduino pin 7
    pinMode(8, INPUT);        // switch is attached to Arduino pin 8
    
    Ethernet.begin(mac, ip);  // initialize Ethernet device
    server.begin();           // start to listen for clients
}

void loop()
{
    EthernetClient client = server.available();  // try to get client

    if (client) {  // got client?
        boolean currentLineIsBlank = true;
        while (client.connected()) {
            if (client.available()) {   // client data available to read
                char c = client.read(); // read 1 byte (character) from client
                // buffer first part of HTTP request in HTTP_req array (string)
                // leave last element in array as 0 to null terminate string (REQ_BUF_SZ - 1)
                if (req_index < (REQ_BUF_SZ - 1)) {
                    HTTP_req[req_index] = c;          // save HTTP request character
                    req_index++;
                }
                // last line of client request is blank and ends with \n
                // respond to client only after last line received
                if (c == '\n' && currentLineIsBlank) {
                    // send a standard http response header
                    client.println("HTTP/1.1 200 OK");
                    // remainder of header follows below, depending on if
                    // web page or XML page is requested
                    // Ajax request - send XML file
                    if (StrContains(HTTP_req, "ajax_inputs")) {
                        // send rest of HTTP header
                        client.println("Content-Type: text/xml");
                        client.println("Connection: keep-alive");
                        client.println();
                        // send XML file containing input states
                        XML_response(client);
                    }
                    else {  // web page request
                        // send rest of HTTP header
                        client.println("Content-Type: text/html");
                        client.println("Connection: keep-alive");
                        client.println();
                        // send web page
                        webFile = SD.open("index.htm");        // open web page file
                        if (webFile) {
                            while(webFile.available()) {
                                client.write(webFile.read()); // send web page to client
                            }
                            webFile.close();
                        }
                    }
                    // display received HTTP request on serial port
                    Serial.print(HTTP_req);
                    // reset buffer index and all buffer elements to 0
                    req_index = 0;
                    StrClear(HTTP_req, REQ_BUF_SZ);
                    break;
                }
                // every line of text received from the client ends with \r\n
                if (c == '\n') {
                    // last character on line of received text
                    // starting new line with next character read
                    currentLineIsBlank = true;
                } 
                else if (c != '\r') {
                    // a text character was received from client
                    currentLineIsBlank = false;
                }
            } // end if (client.available())
        } // end while (client.connected())
        delay(1);      // give the web browser time to receive the data
        client.stop(); // close the connection
    } // end if (client)
}

// send the XML file with switch statuses and analog value
void XML_response(EthernetClient cl)
{
    int analog_val;
    
    cl.print("<?xml version = \"1.0\" ?>");
    cl.print("<inputs>");
    cl.print("<button1>");
    if (digitalRead(7)) {
        cl.print("ON");
    }
    else {
        cl.print("OFF");
    }
    cl.print("</button1>");
    cl.print("<button2>");
    if (digitalRead(8)) {
        cl.print("ON");
    }
    else {
        cl.print("OFF");
    }
    cl.print("</button2>");
    // read analog pin A2
    analog_val = analogRead(2);
    cl.print("<analog1>");
    cl.print(analog_val);
    cl.print("</analog1>");
    cl.print("</inputs>");
}

// sets every element of str to 0 (clears array)
void StrClear(char *str, char length)
{
    for (int i = 0; i < length; i++) {
        str[i] = 0;
    }
}

// searches for the string sfind in the string str
// returns 1 if string found
// returns 0 if string not found
char StrContains(char *str, char *sfind)
{
    char found = 0;
    char index = 0;
    char len;

    len = strlen(str);
    
    if (strlen(sfind) > len) {
        return 0;
    }
    while (index < len) {
        if (str[index] == sfind[found]) {
            found++;
            if (strlen(sfind) == found) {
                return 1;
            }
        }
        else {
            found = 0;
        }
        index++;
    }

    return 0;
}

This sketch is basically a modified version of the sketch from the previous part of this tutorial.

Creating the XML File

The XML_response() function takes care of generating and sending the XML file in the format that has already been explained.

The switches and analog values are inserted into the XML file and sent to the web browser.

HTTP Response

Because the HTTP response must send a different content type for the HTML page and XML file (text/html or text/xml), it has been split up in the sketch to send the correct file type in each HTTP header.

As with the previous part of this tutorial, the web page is stored on the SD card as index.htm and sent when the browser accesses the Arduino web server.

Running the Sketch

Wire up the push button switches and the potentiometer as shown in the circuit diagram from part 9 of this tutorial.

Copy the index.htm file to a micro SD card and insert it into the micro SD card socket of the Ethernet shield. The index.htm file can be copied below.

Load the above sketch to the Arduino and connect to the Arduino with Ethernet shield using a web browser.

You will not seen any significant difference between this part of the tutorial and part 9 of the tutorial, but we now have an easy way of extracting values sent from the Arduino to be used on a web page.

Web Page Source Code

The web page can be copied here and pasted to a file called index.htm:

<!DOCTYPE html>
<html>
    <head>
        <title>Arduino SD Card Web Page using Ajax with XML</title>
        <script>
        function GetArduinoInputs()
        {
            nocache = "&nocache=" + Math.random() * 1000000;
            var request = new XMLHttpRequest();
            request.onreadystatechange = function()
            {
                if (this.readyState == 4) {
                    if (this.status == 200) {
                        if (this.responseXML != null) {
                            // extract XML data from XML file (containing switch states and analog value)
                            document.getElementById("input1").innerHTML =
                                this.responseXML.getElementsByTagName('button1')[0].childNodes[0].nodeValue;
                            document.getElementById("input2").innerHTML =
                                this.responseXML.getElementsByTagName('button2')[0].childNodes[0].nodeValue;
                            document.getElementById("input3").innerHTML =
                                this.responseXML.getElementsByTagName('analog1')[0].childNodes[0].nodeValue;
                        }
                    }
                }
            }
            request.open("GET", "ajax_inputs" + nocache, true);
            request.send(null);
            setTimeout('GetArduinoInputs()', 1000);
        }
    </script>
    </head>
    <body onload="GetArduinoInputs()">
        <h1>Arduino Inputs from SD Card Web Page using Ajax with XML</h1>
        <p>Button 1 (pin 7): <span id="input1">...</span></p>
        <p>Button 2 (pin 8): <span id="input2">...</span></p>
        <p>Analog (A2): <span id="input3">...</span></p>
    </body>
</html>