The image rule no longer passes -p, so fileadder fell through to printing
html_data.h and html_data.c on stdout on every image build. The listing is
only useful when fileadder is run without an image, to see what it would
generate.
Two things kept the same source from producing the same image.
fileadder walked the input directory with readdir(), which returns entries
in whatever order the filesystem hands them over. The same tree therefore
embedded the files in a different order on a different filesystem, and the
image differed byte for byte. scandir() with alphasort() fixes the order.
version.h was listed in .PHONY, which makes it permanently out of date and
propagates that to everything including it, so a plain build recompiled the
four largest translation units even when nothing had changed. Dropping it
from .PHONY alone would go too far the other way: with no prerequisites the
recipe would never run again and the version string would freeze at its
first value. A FORCE prerequisite keeps the recipe running every time, and
the generated header is only moved into place when its content actually
differs, so the timestamp survives an unchanged rebuild.
Measured on this tree: with FORCE the version follows GIT_VERSION and an
unchanged rebuild leaves version.h untouched; without it the header keeps
the value from the first build.
Building with SDCC 4.6.0 emits tons of warnings e.g.:
cmd_parser.c:31: warning 356: object in read-only code space should be const
cmd_parser.c:36: warning 357: pointer to object in read-only code space should be pointer to const
These of course become fatal when building with --Werror.
Comparing resultant binaries, it seems that a single string is shifted
(deduplication?) but the numbers emitted at the end of the build are
otherwise identical.
replaceCalls() ended with strlen(&buffer[pos]), so the length recorded for a
file was the distance to its first 0x00 byte rather than the file size. The
byte just past the content happens to be zero today, which is the only reason
the numbers come out right.
Nothing in html/ contains a 0x00 today, so this is latent, but it is a trap
waiting for the first gzipped or binary asset: gzip puts a zero in the fourth
byte of its header. A gzipped style.css lands in the index as 3 bytes, and
since addr advances by that same value the next file is placed 3 bytes in and
overwrites what came before it. Two gzipped files in html/ are enough to see
it: both come out as 3 bytes and the second sits 1356 bytes inside the first.
Pass the length in and return the adjusted one. The substitution shortens
#{function} to #{NNN}, so the delta is known without measuring. The loops are
bounded by the length as well, which also covers -n, where nothing had put a
terminator in the buffer for strlen to find in the first place.
No output change: the image built from an unmodified html/ is byte for byte
the same before and after, and html_data.h is identical.
Two build-time steps shrink the embedded Web UI (html/) from 113,015 to
34,784 bytes (30.8 %):
- tools/minify.py (new): string-literal-safe minifier that strips
comments and indentation; comments are pure overhead on the wire, so
the transfer size drops already before gzip (113,015 -> 101,498).
Binary files (.ico) are copied through unchanged. The output goes to
output/html_min/, the new fileadder input.
- fileadder -z: gzip-compresses every added file (zlib gzip format,
Z_BEST_COMPRESSION); the generated f_data entries carry a gzip flag.
tools/Makefile links the tool against -lz.
- httpd: adds "Content-Encoding: gzip" for flagged entries; the 8051
only copies compressed bytes, decompression happens in the browser.
The JSON API stays uncompressed.
Fixes required to make the sizes deterministic:
- fileadder terminated files at data_read + 1, so the strlen()-based
size computation depended on uninitialised buffer content; the NUL
now lands directly after the content.
- version.h was .PHONY, so every build rewrote it while parallel builds
could read a half-written copy.
- the file list was unscoped and missed .css/.ico; it is now scoped to
html/ and covers all embedded types.
Measured with make MACHINE=KP_9000_6XHML_X2: data block end moves from
0x5b977 to 0x487e0 (~78 KB freed on a 512 KB image). Verified with
MACHINE=PCB_K0402WS_V3 and on real hardware (PCB-K0402WS-V3).
flash_read_bulk() is kind of a printf function. But did not work as well.
flash_read_bulk() was used in the http generation so this needed to be changed as well.
For the website we decided that we are going to use CSR(Client Side Rendering).
So every HTML and JS files are now static.
Variables are fetched via json calls and output is render on the client side.
Also usefull for the future when we want a REST like API.
execute_config() was also using flash_read_bulk().
This have been replaced with flash_read_bulk() and parsing the flash_buf buffer.
DrDoof
Matt Merhar
René van Dorst
eraiza0816
logicog