How To Deliver Larger Web Pages With An ESP8266

The ESP8266 is a low-cost Wi-Fi chip with full TCP/IP stack and microcontroller unit which makes it very attractive for small DIY IoT projects. When building a web server with the ESP8266, I stumbled over the problem of the limited message size of WiFiClients print() method (It is somewhere around 2922 bytes). Delivering more complex web pages, containing CSS and JavaScript, requires working around the limited memory of the device.

In my last project, I built a HTML template with inline CSS and JavaScript. The template contains HTML comments which are used as markers for a simple Ruby script.

<!-- HEAD -->
...
<!-- SCRIPT -->  
...
<!-- STYLE -->      
...

The script cuts the HTML template at these marked lines and creates a C++ header file.

#!/usr/bin/ruby

require "yui/compressor"

TEMPLATING_RULES = [
  ["\"", "'"]
]

OUTPUT = File.open('../src/ConfigurationTemplate.h', 'w')

def process_html
    f = File.open 'configuration-template.html', 'r'
    compressor = YUI::CssCompressor.new
    
    OUTPUT.puts '//'
    OUTPUT.puts '// Do not edit this file, generated code.'
    OUTPUT.puts '//'
    OUTPUT.puts ''
    
    text = nil
    f.each_line do |line|
        if line.start_with? '<!--'
            if text
                OUTPUT.puts text + '";'
            end
            text = ''
            text += 'const char HTTP_' + line.split(' ')[1] + '[] PROGMEM = "'
        else
          TEMPLATING_RULES.each { |rule|
            line.gsub! rule[0], rule[1]
          }
          text += compressor.compress line
      end 
   end
   OUTPUT.close
end

process_html()

In the header file, the generated Strings, which can be very long, are stored using the PROGMEM() macro. PROGMEM data is stored in flash (program) memory instead of the SRAM. On Arduino and ESP8266 a variable such as const char * will be placed in RAM, not flash memory. It is possible to place this variable into flash memory and load it when it is needed into RAM.

const char HTTP_HEAD[] PROGMEM = "<!DOCTYPE html><html><head lang='en'><title>My Title</title>";

In the web server, I had to concatenate the flash Strings to one HTTP response. For this case I used the FSPTR() macro. FSPTR takes a PROGMEM pointer and casts it to __FlashStringHelper to pass it to other functions.

server = new ESP8266WebServer(serverPort);
...
String page = FPSTR(HTTP_HEAD);
page += FPSTR(HTTP_SCRIPT);
page += FPSTR(HTTP_STYLE);
String body = FPSTR(HTTP_BODY);
... 
page += FPSTR(HTTP_FOOT);
... 
server->sendHeader("Content-Length", String(page.length()));
server->send(200, "text/html", page);

Without the ability to add runtime data the web page would be useless. I used simple String replacement to achieve this. In my template, I used the placeholder syntax {{property name}} to mark which parts should be replaced. Here is an example:

<p class="text">Firmware: <b>{{firmware}}</b></p>

The web server replaces these placeholders when creating the HTTP response via String.replace() at runtime.

String body = FPSTR(HTTP_BODY);
body.replace("{{firmware}}", configuration.firmware);
page += body;

With this setup it is possible to deliver bigger web pages even with the limited RAM of an ESP8266. You can find the complete project on GitHub.

Reference: PROGMEM and FSPTR documentation

WRITTEN BY

Sebastian Glahn is a Senior Software Engineer living in Cologne. He writes about Software Development, 3D-Printing, Robots and other stuff. He is also a maintainer of several open source projects.

about | github | twitter