The two-hour silence
Every hardware project has a day when the data goes weird and then stops. Ours came with a plot twist: the sensor wasn't lying, and the crash wasn't a crash.
First, the weird data
For a day or two the wave heights had been wrong in a way that made no sense. Phantom metres on a sea that people on the beach called moderate. We chased firmware theories, calibration theories, interference theories. Then I went out to the buoy and found the answer floating right next to it: the solar panel had come out of the lid and was hanging in the water on its cable. Every little tug of that cable was hauling the electronics box around inside the hull. The motion sensor had been reporting motion perfectly honestly. It was just measuring the panel's tantrums, not the sea.
Then, the silence
I pressed the panel back into its seat, and she went quiet. Two hours, nothing. Not a watchdog reset, not a crash-and-recover, nothing on the wire at all. That is the fingerprint of a power interruption, the one kind of event no software can log while it is happening, because the computer doing the logging has lost power too. Somewhere in the fiddling, the battery protection had done exactly what it is built to do: cut everything off and ask questions never.
The fix was the oldest one in engineering. The power switch, off and on. She booted clean, reported a fresh boot count and a wiped clock (the tell-tale signs of a true power loss rather than a software fault), and got on with her day as if nothing had happened.
What we changed
- The panel seating is now a checklist item. Cable slack, seat pressure, lid torque, all checked before the hull ever closes again.
- Boot forensics got richer. Every check-in already carries the boot reason, so the difference between a normal wake, a watchdog reset and a cold power-on is now the difference between three plain words. Two hours of silence followed by a power-on boot tells the whole story in two fields.
- Respect for the humble power cycle. A sealed instrument has to survive its power being yanked at any instant, and it turns out the best test of that is reality administering it, unannounced.
The pattern of this whole project keeps repeating: the sensor was honest, the protection circuit was honest, and the failure was mechanical. The electronics keep turning out to be the most truthful witnesses we have.