Skip to content

Knowledge Base

10 Topics 10 Posts

General embedded, IoT, and programming knowledge — beyond the course.

This category can be followed from the open social web via the handle knowledge-base@forum.grapesiot.com

Subcategories


  • Patterns, idioms, common bugs, best practices in C for embedded

    2 2
    2 Topics
    2 Posts
    A
    Hardware speaks in bits, and C is how you answer. Nearly every peripheral bring-up is the same loop: read the register map row, build the mask, write it, read it back to verify. Engineers who are fluent in that loop bring up drivers in an afternoon; everyone else spends a week toggling the wrong bit. The core skills Bit manipulation as a first language. Setting, clearing, toggling and testing individual bits with shifts and masks — REG |= (1 << EN), REG &= ~(3 << MODE), if (REG & READY) — is the vocabulary of every driver you'll ever write. Practice until masks feel like plain reading. → Bit manipulation and register programming → Hands-on bit manipulation exercises Structs as register maps and sensor data. A well-laid-out struct overlays a peripheral's registers or packs a sensor reading (SensorReading_t with temperature, humidity, timestamp) into something every layer of firmware can share. Unions pull double duty for protocol payloads where the same bytes mean different things by message type. → Functions, structs, unions and macros Functions, macros and the HAL habit. Raw register writes scattered through application code rot fast. A thin hardware-abstraction layer — sensor_init(), sensor_read(), one place per peripheral — means a pin change or a sensor swap touches one file, not forty. Know when a macro beats a function (and when it hides a double-evaluation bug). Headers and sources that scale. One giant main.c dies at a few hundred lines. Split by peripheral and by concern, guard every header, keep ISRs thin and fast, and let the build tell you about mistakes with warnings-as-errors before the hardware does. Start at the Embedded C deep track for the full six-part treatment. Share a driver snippet, a mask trick, or a HAL layout that survived contact with real hardware. Tag with c-lang.
  • Branching strategy, tagging releases, .gitignore, secrets management

    1 1
    1 Topics
    1 Posts
    A
    When a device fails in the field, the first question is: "what exact binary is on it, and can we rebuild it?" If your version control can't answer that, nothing else in your workflow matters. Checklist before you ask for help here Every shipped binary maps to a tag (git tag v1.2.3), not "final_final2". A tag is the only way to rebuild byte-identical firmware months later when a customer reports a bug on old hardware. → Tags, releases and repository hygiene Build artefacts, secrets and calibration dumps are ignored. .pio/, build/, certificates, tokens and per-device calibration data never belong in history. Once a secret is committed, rotating it is the only fix — deleting the file later doesn't remove it from history. No credentials in the repo. Ever. Device tokens and WiFi passwords live in provisioning, environment, or a secrets store — not in a header file someone will push to GitHub. You can bisect back to the breakage. When a regression appears three weeks late, git bisect with small, honest commits finds the culprit in minutes. "Backup Friday" commits make that impossible. → Advanced workflows and recovery Start at Git & Version Control and work through it in order. Share your workflows, ignore-file setups, or mistakes learned the hard way. Tag with git.
  • CLI tools, bash scripting, SSH, Docker basics, file system navigation

    1 1
    1 Topics
    1 Posts
    A
    You don't need to become a sysadmin. You need the slice of Linux that unblocks firmware work: moving fast, reading logs, reaching a remote machine, and automating the boring flash-and-test loop. Recipes you'll use weekly Hunting through logs. A device prints thousands of lines and the one error matters. grep -i "error\|panic\|wdt" serial.log finds it; piping build output through a filter beats scrolling. Learn pipes and redirection once and every future debugging gets faster. → Searching, transforming and inspecting data → Shell basics and text processing Talking to serial from the terminal. screen /dev/ttyUSB0 115200 (or minicom) gets you a monitor without opening an IDE. When a port disappears after a reboot, knowing how to list devices and check permissions saves an hour of replugging cables. → Permissions, users and processes Reaching remote machines. SSH into a build server or a gateway in the field, copy firmware over with scp, forward a port to see a remote dashboard locally. Most "the server is broken" moments are a wrong path, a dead service, or a full disk — all diagnosable over SSH. Automating the loop. Flashing ten boards by hand is how mistakes happen. A five-line shell loop that builds, flashes, and captures each board's first minute of logs turns a painful afternoon into a coffee break. → Pipes, processes and automation Start at Linux Fundamentals for the OS side, then the command-line tools above for speed. Post your useful one-liners, serial-port recipes, SSH quirks, or WSL fixes. Tag with linux or terminal.
  • TimescaleDB, PostgreSQL basics, SQL queries, time-series data concepts

    1 1
    1 Topics
    1 Posts
    A
    One device sending telemetry every 5 seconds produces about 17,000 rows a day. A hundred devices produce 1.7 million rows a day — all timestamped, all needing fast answers for dashboards and alerts: average temperature per hour, last-seen per device, events inside a time window. This is where most IoT projects quietly succeed or fail, long after the firmware works. Start with Postgres, think in time Postgres handles millions to low billions of rows on ordinary hardware when the schema is narrow, indexed on (device_id, ts), and old data is dropped by time range. The discipline that makes it scale — and yes, this is a data-modelling problem first, which is why it helps to have the fundamentals straight: → Database fundamentals: modelling and normalisation One time column to rule queries. Every analytical query bounds ts. Queries that filter on payload alone (WHERE temperature_c > 30, no time bound) are full-table scans at any scale — always add the time bound first. Narrow rows, typed columns. Keep hot fields relational. Unmodelled JSON accumulates five spellings of temperature and breaks every downstream query — and columnar compression works best on typed columns, not opaque JSON. Retention is a partition operation. Dropping whole time partitions by age runs in milliseconds; deleting a year row-by-row can stall a cluster for days. Design deletes by time range from day one. → Native partitioning, compression thinking and retention Precompute the dashboards. An hourly rollup table turns a million-row dashboard scan into a 24-row lookup. Same GROUP BY date_trunc shape, precomputed — keep the bucket expression identical in raw and rollup form. → Joins, aggregation and subqueries The Timescale upgrade path (concepts, no new queries) When the fleet grows, TimescaleDB automates exactly these patterns with zero query rewrites — because your queries already bound ts and group by (device_id, date_trunc): Hypertables auto-create time chunks behind a normal-looking table (create_hypertable on ts), with per-chunk indexing and pruning that behaves like your manual partitions. Compression on chunks older than N days (roughly 10x on narrow numeric telemetry) for cheap long retention. Continuous aggregates replace hand-maintained hourly tables with background materialisation. Retention policies (add_retention_policy, e.g. raw 90 days, rollups 2 years) replace the detach-and-drop routine. The rule from the field: earn the extension with growth numbers. A Timescale dependency for a million rows loses to a clean native design with pruning, two composite indexes, and a documented upgrade path. Finish with the telemetry schema capstone — a 500-sensor fleet designed and measured end to end. Post your schema, the slow query with its EXPLAIN, and the time range involved. Tag with database.
  • HTTP methods, status codes, JSON parsing on device and server side

    1 1
    1 Topics
    1 Posts
    A
    Devices speak MQTT. Everything else — dashboards, scripts, platform config, phone apps — speaks REST and JSON. You'll meet this layer the moment a webhook never arrives, a platform call returns 403, or firmware chokes on a field that's suddenly null. Three stories you'll live through "It works in Postman but not on the device." Postman hides memory limits. On an ESP32, parsing a 4 KB JSON payload with the wrong strategy fragments the heap or blows the stack. Stream or bound your parsing, check every field's presence and type before use, and never assume a key exists because it did yesterday. The device side has its own rules — when to reach for REST over MQTT, how to keep the memory footprint flat: → HTTP and REST from the device side "The API returns 403 and I don't know why." Almost always auth scope, an expired token, or a path that doesn't match your device ID exactly. Read the status code properly — 401 means prove who you are, 403 means you proved it and still can't, 429 means slow down — then reproduce with curl before touching firmware. The underlying principles are worth learning once, properly: → HTTP and REST architectural principles → Validation, errors and authentication "The payload shape keeps changing." Firmware v1 sends this: { "t": "v1", "temp": 27.4, "hum": 62.1 } Firmware v2 sends this: { "t": "v2", "temperature_c": 27.4, "humidity_rh": 62.1, "vbat_mv": 3910 } The dashboard breaks for half the fleet. Contracts matter: version your payloads (t or v on every message), accept missing optional fields with defaults, never crash on extra fields, and reject unknown major versions with a counted alert instead of silent garbage. Design the shape deliberately up front and this class of bug disappears: → Resource modelling and API contracts Symptom → what to learn Symptom What to learn 401/403 from a platform API Auth, tokens and scopes Payload too big or shape drifting Resource modelling and contracts Works on desktop, fails on ESP32 JSON parsing under RAM limits Start at REST API Development and work through it in order. Post the method + URL + status code + body (redact secrets), and what curl says. Tag with api or json.
  • Threat models, certificate management, secure boot, real attack case studies

    2 2
    2 Topics
    2 Posts
    A
    Knowing your threat model is half the job. This companion post is the other half: the concrete mechanisms that keep a deployed fleet trustworthy for years — identity, storage, boot integrity, and updates that can't brick everything at once. One device, one credential Fleet-wide passwords and hardcoded API keys are a supply-chain vulnerability — one leak compromises every unit ever shipped. Each device gets unique credentials issued at provisioning, with expiry and rotation planned from day one. Certificates beat passwords where the hardware allows it: a client certificate in protected storage is harder to phish, share, or guess than any string a human invents. → Certificate-based auth and device tokens Secrets live in protected storage, not source code On-device credentials belong in NVS with proper access control plus a provisioning flow — never as string literals in firmware. Assume anyone can dump your flash; design so a dump alone isn't enough. Work through what happens when a unit is returned, resold, or stolen: wipe procedures and revocation lists are part of the design, not an afterthought. The general principle — shrink what the device exposes — applies far beyond storage: → Credential storage on the device → Device security: shrinking the attack surface Trust the code you run Secure boot stops modified firmware from executing on your hardware. → The secure boot concept Signed OTA with hash verification stops tampered images from installing. Automatic rollback on failure stops one bad release from becoming a fleet-wide outage — an update path without rollback is a disaster waiting for a trigger. The full identity-to-update story, end to end: → Secure OTA with signed firmware → Identity, authorisation and secure updates What "passing review" looks like Unique per-device credentials (never hardcoded), TLS with proper certificate validation and clock sync, secure boot plus signed OTA with rollback, and a tested incident routine. That's the bar that separates professional firmware from hobbyist firmware. Finish with IoT security engineering — threat modelling to incident response in one track. Post what you've hardened, your provisioning flow, or your OTA rollback story. Tag with security.
  • Digital filters, FFT concepts, fixed-point arithmetic, averaging methods

    1 1
    1 Topics
    1 Posts
    A
    Most firmware projects hit a wall that only maths solves — a sensor reading too noisy to use, a frequency that must be detected, a filter that needs tuning, a calculation that loses precision on a microcontroller with no FPU. This post is the practical core: no academic DSP, just the techniques that come up with real sensors. Sample fast enough, or lie to yourself Nyquist: to represent bandwidth B you need fs > 2B; anything above fs/2 folds back as a phantom: f_alias = |f - k*fs|. Thermal logger at fs = 10 Hz with 60 Hz mains hum coupled into the wiring → k=6 → |60-60| = 0 Hz. Your "temperature drift" is mains hum pretending to be DC. 100 Hz sampler with 130 Hz motor PWM residue → |130-100| = 30 Hz phantom. You chase a resonance that doesn't exist. The engineering rule (not the textbook 2x): sample at 5–10x the highest frequency you must resolve, so your filter has room to roll off. Averaging after aliasing can't help — the fold already happened at the sampling instant. Anti-alias in analog (RC before the ADC) or by oversampling-then-filtering at high fs. Smooth with the right tool Moving average (N): white noise drops by √N, lag ≈ N·T/2. General thermal and ADC cleanup (N = 16–64). Exponential (EMA): y += alpha*(x-y), O(1) RAM. Display smoothing and tight loops. Median (odd N): kills spikes (WiFi bursts, brush clicks), preserves edges. Put it before averaging in spiky environments. Oversample + decimate: 4x oversampling buys roughly 1 extra bit (16x → +2 bits: sample 160 Hz, average 16, output 10 Hz). Needs real noise across 1 LSB to dither. Hysteresis/deadband: act only past ±band. Stops relay and fan chatter around thresholds. → Sampling and signal conditioning for industrial sensors Convert counts to units — with proof Two-point calibration: take two raw/reference pairs, fit T = a·counts + b, then validate with a third point and record references, conditions, date, and stale-data behaviour (hold-last-good, never actuate on unvalidated samples). Sample on a stable timebase (millis() cadence with +=, not delay() drift under WiFi load). Takeaways: match the rate to the physics, filter in analog before digital, median-before-average for spikes, calibrate with numbers and an expiry date. Finish with Sampling, noise and calibration — the full treatment this primer is distilled from. Post your sensor, your rate, the filter you tried, and the numbers. Tag with signal-processing.
  • How to extract relevant information, register map reading, timing diagrams

    1 1
    1 Topics
    1 Posts
    A
    A datasheet is the ground truth for every component you use. Guessing at voltage limits, timing requirements, or register values is how you fry a board or spend a week debugging something the datasheet would have told you in ten minutes. The skill isn't reading every page — it's knowing which pages matter, in what order. The structure of these documents is a skill of its own: → How technical documents are structured Read in this order Absolute Maximum Ratings — what kills the part. 5V into a 3.3V GPIO is a gamble, not a spec. Electrical Characteristics — the voltages, currents and timings you actually design to. This is where requirements turn into numbers you can hold a design to: → Extracting requirements and electrical limits Register map — the exact bit that enables the interrupt, sets the mode, or clears the flag. Timing diagrams — setup and hold, clock polarity, what the signal must look like on a scope. Register rows and timing plots are where firmware meets silicon: → Register maps, timing and protocol details App note + errata — how the vendor says to use the part, and what they corrected after printing. A worked 60-second example You need to enable a sensor's data-ready interrupt. The register map shows INT_ENABLE at address 0x2D, bit 3, "write 1 to enable, cleared on read of STATUS". That one row tells you the address, the bitmask (1 << 3), the write semantics, and the clearing rule — four facts you'd otherwise discover by trial and error. Every peripheral bring-up is this loop: map row → mask → write → verify by reading back. → Hands-on intro to reading datasheets Start at Reading Datasheets and work through it in order. Post the part number, the page/section, and what confuses you. Tag with datasheets.