Correct wire format to disassembly: raw S1||crypt(S2), radix 32, int snno/seconds, decryptData response parser, --probe

- envelope: 7 fixed keys + conditional dataArea, no data key; int
  type/snno, seconds timestamp, data setter auto len/crc (CRC32 upper);
  SequenceNumber 1..9999 (increaseSN)
- crypto: encrypt is plaintext-header-first a||transform(b), key radix 32
  at both tryParse sites; decrypt_data ports decryptData (dual-0x58
  markers, s+'}' equipNo probe, s+','+T return); decrypt_request kept
  for the device-side known-header split
- socket: raw send (no length prefix), accumulate-and-try receive
- battery: queryAPowerData shape (data={'update':2}), radix default 32,
  --probe prints exact TX/RX bytes per variant; device_local.send_probe
- docs: RE_DART_FINDINGS ss3/5/8 + open items, BATTERY_POC host recipe
- tests: 19 new protocol tests, suite 44/44 green

docs/BATTERY_POC.md

@@ -19,11 +19,17 @@ see `RE_DART_FINDINGS.md` §§3–5, 8, 10.
 
 1. Probes TCP ports `9000, 18000` in order (`9000` was open on the tested
    unit; `0x4650 = 18000` is the app constant but was closed there).
-2. Sends `DEVICE_APOWER_INFO (0x19)`, then `BMS (0x1D)`, `EMS (0x1E)`:
+2. Sends `DEVICE_APOWER_INFO (0x19)` with `data={"update": 2}` (exactly what
-   compact-JSON `CommunicationObject` → `encrypt_object` (key
+   the app's `queryAPowerData` passes as `arguments`), then `BMS (0x1D)`,
-   `hex(sum(sn) & 0xFF)`) → int16LE length framing.
+   `EMS (0x1E)`: compact-JSON `CommunicationObject`
-3. Decrypts the reply and extracts `currentBatterySoc` → `soc` →
+   (`cmdType, equipNo, type, timeStamp, snno, len, crc[, dataArea]`, no
-   `avg(socArray)`.
+   `data` key) → `encrypt_object` (`S1` plaintext header first, tail
+   `(byte + abs_index + n) & 0xFF` with `n = int(key, 32)`) → sent **raw**
+   over the socket (no length prefix — `ClientSocket.send` forwards to
+   `_Socket.add` directly).
+3. Accumulates reply chunks and runs the app's `decryptData` logic (second
+   `0x58` marker → `s + "}"` equipNo probe → tail decode → `s + "," + T`),
+   then extracts `currentBatterySoc` → `soc` → `avg(socArray)`.
 
 ## Usage
 
@@ -36,11 +42,14 @@ python -m franklinwh_agate_client.battery --sn <20-char-SN>
 
 # full reply envelope
 python -m franklinwh_agate_client.battery --sn <20-char-SN> --raw
+
+# low-level probe: exact TX bytes + exact RX bytes per request variant
+python -m franklinwh_agate_client.battery --sn <20-char-SN> --probe
 ```
 
 Options: `--host` (default `10.100.1.1`), `--ports 9000,18000`,
-`--key-radix {16,32}` (default 16; 32 is the alternate candidate in the
+`--key-radix {16,32}` (default 32, per the `radix: 32` literal at both
-disassembly), `--timeout`.
+`tryParse` sites), `--timeout`.
 
 Programmatic:
 
@@ -50,9 +59,31 @@ out = get_battery("...", host="10.100.1.1")
 print(out["value"], out["source"])  # e.g. 78.0 currentBatterySoc
 ```
 
+## Host probe recipe (run on the machine joined to the device AP)
+
+The agent container cannot reach `10.100.1.x`; run this on the host:
+
+```bash
+cd franklinwh-agate-client
+python3 -m franklinwh_agate_client.battery \
+  --sn A03X2516013210060005 --host 10.100.1.1 --ports 9000,18000 --probe
+```
+
+Expected: `secret key for ...: 1c (radix 32)`, then per-variant `tx`
+(encrypted envelope, starts with the ASCII header
+`{"cmdType":25,"equipNo":"A03X2516013210060005",`) and `rx` bytes.
+A rejected frame is met with zero bytes + close; any non-empty `rx` is
+the device answering — paste the `rx` hex back and the reply parses via
+`decrypt_data`.
+
+If every variant still returns zero bytes, capture one app-driven session
+instead: install the Frida hook (`blutter_out/blutter_frida.js` pattern —
+log `Socket Manager send object: ` / `receive object: `), tap *Refresh* in
+the app while joined to the AP, and compare its TX bytes against ours.
+
 ## Limits
 
 - No live round-trip has succeeded yet from an isolated container netns
   (all probes `ECONNREFUSED`); first success needs a `10.100.1.x` source.
-- `key_radix`, BLE field order, and `X-Device-*` binding remain open
+- BLE field order and `X-Device-*` binding remain open
   (`RE_DART_FINDINGS.md` Open items).

docs/RE_DART_FINDINGS.md

@@ -77,20 +77,36 @@ esp_provisioning_ble/src/...                                            # ESP pr
 
 ## 3. Wire envelope — `CommunicationObject`
 
-JSON keys, in the exact insertion/inverse order used by `toJson`:
+JSON keys, in the exact insertion order used by `toJson`:
 
 ```
-cmdType, equipNo, type, timeStamp, snno, len, crc, dataArea, data
+cmdType, equipNo, type, timeStamp, snno, len, crc[, dataArea]
 ```
 
-Fields present: `_Mint field_8`, `_OneByteString field_10`, `_Mint field_14`
+`dataArea` is appended **only when the payload is present and non-empty**;
-(+ `_Mint`/`_OneByteString` further). `data` is a setter; string const `"00000000"` is a
+there is **no** `data` key on the wire. `cmdType` = `CommunicationCmd` raw,
-default (likely `crc`/`dataArea` placeholder). `cmdType` = `CommunicationCmd` raw,
+`type` = `CommunicationOperate` raw (always 0 on the `communicate`-factory
-`type` = `CommunicationOperate` raw, `equipNo` = device SN.
+path — the ctor zeroes it), `equipNo` = device SN (`"00000000"` fallback
+when empty), `timeStamp` = whole **seconds** (`micros ~/ 1000 ~/ 1000`),
+`snno` = `int` sequence (see below, **not** the serial string), `len` =
+UTF-8 byte length of the compact JSON of the payload, `crc` =
+`Crc32.calculate(jsonEncode(data))` as **uppercase** unpadded hex.
+Null/empty payload ⇒ `len == 0`, `crc == ""`, no `dataArea` key.
 
 - `CommunicationOperate`: `query=0, set=1, command=2, modify=5`.
 - `Crc32::calculate` is used in `fromJsonString` (integrity check of the payload).
-- `toJsonString()` produces the string that is then transformed (see §5).
+- `toJsonString()` = `jsonEncode(toJson())` — compact JSON, the input of
+  `encryptObject` (see §5). `toString()` likewise returns `toJsonString()`.
+- `CommunicationObject(cmd, equipNo, {data})` ctor zeroes `type`/`snno`,
+  stamps seconds, and routes `data` through the `data=` setter (which
+  recomputes `len`/`crc`).
+- `sendObject` stamps `snno = increaseSN()` first: a 1-based counter that
+  wraps to 1 past 9999 (manager field starts at 0, so the first request
+  carries `snno == 1`).
+- The `communicate` factory (`0xc9264c`) builds the `data` map from the
+  caller's `arguments` (e.g. `queryAPowerData` passes `{"update": 2}` with
+  cmd `deviceApowerInfo`/`0x19`, operate `query`) and calls the ctor with
+  a named `data:` argument.
 
 ## 4. `CommunicationCmd` (raw → name)
 
@@ -119,20 +135,37 @@ default (likely `crc`/`dataArea` placeholder). `cmdType` = `CommunicationCmd` ra
 ## 5. Encryption — `communication_center/core/encryption.dart`
 
 **`generateSecretKey(String equipNo)`** = hex of `(sum(utf8(equipNo)) & 0xFF)` (via
-`reduce(sum)` → `&0xFF` → `_toPow2String(16)`), i.e. a 1-byte key rendered as hex.
+`reduce(sum)` → `&0xFF` → `_toPow2String(16)`), i.e. a 1-byte key rendered as hex,
+unpadded. E.g. SN `A03X2516013210060005` → byte sum `1052 & 0xFF = 28` → `"1c"`.
 
-**`encryptObject(obj, key)`**:
+**`encryptObject(object, key)`** (object in arg slot 0, key in slot 1):
-1. `s = obj.toJsonString()`; split on `","`.
+1. `s = object.toJsonString()`; split on `","`.
-2. `a` = UTF-8 of `part[0] + "," + part[1]`  (i.e. `{"cmdType":..` and `"equipNo":..`).
+2. `S1 = parts[0] + "," + parts[1] + ","` (header, *with* trailing comma);
-3. `b` = UTF-8 of `join(",", parts[2..])` (the rest).
+   `a = UTF-8(S1)`.
-4. `keyInt = int.tryParse(key, radix: …)`.
+3. `S2 = join(",", parts[2..])` (the rest); `b = UTF-8(S2)`.
-5. `out[i] = (b[i] + i + len(a) + keyInt) & 0xFF` for all i, then append `a` **unchanged**.
+4. `keyInt = int.tryParse(key, radix: 32)` (literal `0x20` at the call site;
+   null → 0).
+5. `out[i] = (b[i] + i + len(a) + keyInt) & 0xFF` for all i (so each tail
+   byte is shifted by its *absolute* JSON index plus the key).
+6. Result = `a ++ out` — the plaintext header goes on the wire **first**,
+   transformed tail after (`addAll` onto the header copy).
 
-So on the wire the encrypted body is `transform(b) || a` (header/keys appended at the end).
+**`decryptData(bytes)`** (single arg, the accumulated receive buffer):
-`decryptData` is the inverse.
+1. `i = indexWhere(byte == 0x58)`, `j = indexWhere(byte == 0x58, start: i+1)`;
+   either missing → return `""`.
+2. `s = utf8(wire[0..j])`; `probe = jsonDecode(s + "}")`, null → `""`.
+3. `equipNo = probe["equipNo"]`; `key = generateSecretKey(equipNo)`;
+   `keyInt = int.tryParse(key, radix: 32) ?? 0`.
+4. `tail = wire[j+1..]`; `out[k] = (tail[k] - k - len(utf8(s + ",")) - keyInt) & 0xFF`.
+5. Return `s + "," + utf8(out)` (the `","` re-inserts the comma the `0x58`
+   separator replaced). The caller (`_onSocketReceiveData`) `jsonDecode`s
+   that, builds `CommunicationObject.fromJson`, logs
+   `Socket Manager receive object: …`, and clears the buffer; `""`/parse
+   failures keep accumulating ("receive incomplete data").
 
-> This is a bespoke additive/KSA stream; the exact radix of the key parse and the trailing
+So requests are `S1 || transform(S2)` (split at the computable header
-> `a` handling should be confirmed against a captured frame before trusting it byte-for-byte.
+length) while responses are `s || 0x58 || transform(tail)` (split at the
+second `0x58` marker). Both halves share the one additive stream.
 
 ## 6. Checksums & byte order
 
@@ -191,7 +224,16 @@ authS2C=2    "fwh/s2c/auth/"  mngC2S=5   "fwh/c2s/mng/"     deviceS2C  "fwh/s2c/
   - host = stored host, else **`"10.100.1.1"`** (device hotspot gateway);
   - port constant **`0x4650 = 18000`** (candidate local port),
   - a `Duration` and retry loop (up to ~2 retries) are involved.
-- `BytesToInt16/Int16ToBytes` LE framing; `Socket Manager send object` / `receive object`.
+- `sendObject`: stamps `snno = increaseSN()`, logs
+  `Socket Manager send object: <json>`, derives the key with
+  `generateSecretKey(equipNo)`, calls `encryptObject(object, key)`, then
+  `_sendData`, which checks `connected` status and calls `ClientSocket.send`
+  → **`dart:io _Socket.add` with the bytes as-is: raw send, no length
+  prefix, no markers, no handshake** — then clears the receive buffer.
+- Receive: socket `listen` chunk → `ClientSocket._onSocketReceiveData` →
+  `SocketManager._onSocketReceiveData` → buffer `addAll` → `decryptData`
+  in try/catch (`""` = keep accumulating) → `jsonDecode` →
+  `CommunicationObject.fromJson` → `Socket Manager receive object: …` log.
 
 ### Empirical — live aGate on its SoftAP (2026-10-13, from a client on `10.100.1.0/24`)
 
@@ -206,11 +248,11 @@ Full TCP sweep of `10.100.1.1` (ports 1–65535): exactly **two** ports open —
     `16 03` = int16LE `790`, so the decoder waits for a 790-byte frame that never
     completes, then closes on idle;
   - cleartext HTTP `GET` (`47 45` = `17735`) hangs forever (read timeout).
-  - A *complete* 147-byte frame (int16LE length + encrypted `QUERY_DEVICE_INFO`
+  - Earlier probes are superseded: they used an int16LE length prefix (the
-    envelope, `equipNo = X25160132`, key from `sum(utf8(sn))&0xFF`) is accepted
+    socket layer sends **raw**), the old `transform(b) || a` order (correct
-    but met with **zero bytes** — every command tried (`0x00`/`0x01`/`0x02`/`0x06`)
+    is `a || transform(b)` — §5), a truncated `equipNo`, and default radix
-    and both key radices (16/32). Content-level rejection; the leading hypothesis is
+    16. All are corrected in the current PoC; see `BATTERY_POC.md` for the
-    `equipNo` validation against the real serial. **Pending: retry with the true SN.**
+    host probe recipe with the true SN.
 
 ## 9. Bluetooth (BLE) transport — `communication_center/bluetooth`
 
@@ -341,16 +383,22 @@ keepAlive 10 s, LWT, QoS 0, event names), `MethodHandlerManager` handler list,
 Still open:
 
 1. Exact BLE frame field order & the 2-byte magic semantics (confirm with `btsnoop`).
-2. Exact `encryptObject` radix + trailing-header handling.
+2. ~~Exact `encryptObject` radix + trailing-header handling~~ — **resolved
+   from disassembly (see §5): radix 32, plaintext header first
+   (`S1 || transform(S2)`), `decryptData` marker model. Remaining: first
+   confirmed live round-trip.**
 3. ~~Local HTTP port + TLS behavior for `https://IP:PORT`~~ — **revised (2026-10-13,
    §8 empirical):** no HTTP/TLS port exists on the device at all (only 53 + 9000).
    Either local mode is socket-only on this firmware, or the HTTP service is
    state-dependent and wasn't running. Revisit if a `/terminal` response is ever
    captured from the SoftAP.
-4. ~~Local socket port confirmation~~ — **resolved on one unit (2026-10-13): `9000`**
+4. ~~Local socket port confirmation~~ — **partially resolved: `9000` open on
-   (§8 empirical). The `0x4650` constant is not listening; the port may be
+   one unit** (§8 empirical); the `0x4650` constant is not listening there,
-   firmware-/device-dependent. Remaining: first confirmed RPC round-trip
+   so the port may be firmware-/device-dependent. Remaining: first confirmed
-   (blocked on the real SN — see §8 empirical).
+   RPC round-trip — the corrected request bytes (raw `S1 || transform(S2)`,
+   radix 32, int `snno`, seconds `timeStamp`, true SN
+   `A03X2516013210060005`) are ready via
+   `battery --probe`; needs a host on `10.100.1.0/24`.
 5. Which class in `classes5.dex` sets the `X-Device-*` headers and their exact values
    (string-pool location confirmed; class binding in progress — §13).
 6. (Tooling, optional) Enable REA's DEX tools: obtain a **jadx-headless-mcp 0.7.1** JAR

franklinwh_agate_client/__init__.py

@@ -35,6 +35,7 @@ __all__ = [
     "derive_hotspot_credentials",
     "extract_soc",
     "get_battery",
+    "probe_device",
     "SOCKET_PORT_CANDIDATES",
     "DEVICE_GATEWAY_IP",
     "FALLBACK_PASSWORD",
@@ -45,7 +46,8 @@ __all__ = [
 def __getattr__(name: str):
     # Lazy so `python -m franklinwh_agate_client.battery` does not import
     # the submodule twice (avoids a runpy RuntimeWarning).
-    if name in {"extract_soc", "get_battery", "SOCKET_PORT_CANDIDATES"}:
+    if name in {"extract_soc", "get_battery", "probe_device",
+                  "SOCKET_PORT_CANDIDATES"}:
         from . import battery as _battery
 
         return getattr(_battery, name)

franklinwh_agate_client/battery.py

@@ -125,15 +125,16 @@ def get_battery(
     sn: str,
     host: str = "10.100.1.1",
     ports: Tuple[int, ...] = SOCKET_PORT_CANDIDATES,
-    key_radix: int = 16,
+    key_radix: int = 32,
     timeout: float = 5.0,
 ) -> Dict[str, Any]:
     """Query the aggregate charge percentage over the local TCP socket.
 
-    Tries ``DEVICE_APOWER_INFO`` (0x19) first, then ``BMS`` (0x1D) and
+    Mirrors the app's ``queryAPowerData`` first (``DEVICE_APOWER_INFO`` with
-    ``EMS`` (0x1E) as fallbacks, probing each candidate port. Returns the
+    ``data={"update": 2}``), then ``BMS`` and ``EMS`` as fallbacks, probing
-    ``extract_soc`` dict plus ``cmd``/``port``/``reply`` context. Raises the
+    each candidate port. Returns the ``extract_soc`` dict plus
-    last socket error if no port answers.
+    ``cmd``/``port``/``reply`` context. Raises the last socket error if no
+    port answers.
     """
     last_err: Optional[Exception] = None
     for port in ports:
@@ -141,13 +142,13 @@ def get_battery(
             sn=sn, host=host, socket_port=port,
             key_radix=key_radix, timeout=timeout,
         )
-        for cmd in (
+        for cmd, data in (
-            CommunicationCmd.DEVICE_APOWER_INFO,
+            (CommunicationCmd.DEVICE_APOWER_INFO, {"update": 2}),
-            CommunicationCmd.BMS,
+            (CommunicationCmd.BMS, None),
-            CommunicationCmd.EMS,
+            (CommunicationCmd.EMS, None),
         ):
             try:
-                reply = client.send_rpc(cmd)
+                reply = client.send_rpc(cmd, data=data)
             except (ConnectionError, socket.error, TimeoutError) as exc:
                 last_err = exc
                 log.debug("port %d cmd %s failed: %s", port, cmd, exc)
@@ -164,6 +165,47 @@ def get_battery(
     raise ConnectionError(f"no battery reply via {host}:{list(ports)}") from last_err
 
 
+#: ``--probe`` request variants: (label, cmd, data).  The first mirrors the
+#: app's ``queryAPowerData`` exactly; the rest isolate single envelope
+#: decisions (payload present/empty/absent, fallback commands).
+PROBE_VARIANTS: Tuple[Tuple[str, CommunicationCmd, Any], ...] = (
+    ("apower/update2", CommunicationCmd.DEVICE_APOWER_INFO, {"update": 2}),
+    ("apower/none", CommunicationCmd.DEVICE_APOWER_INFO, None),
+    ("apower/empty", CommunicationCmd.DEVICE_APOWER_INFO, {}),
+    ("queryDeviceInfo/none", CommunicationCmd.QUERY_DEVICE_INFO, None),
+    ("bms/none", CommunicationCmd.BMS, None),
+    ("ems/none", CommunicationCmd.EMS, None),
+)
+
+
+def probe_device(
+    sn: str,
+    host: str = "10.100.1.1",
+    ports: Tuple[int, ...] = SOCKET_PORT_CANDIDATES,
+    key_radix: int = 32,
+    timeout: float = 5.0,
+) -> List[Dict[str, Any]]:
+    """Send every :data:`PROBE_VARIANTS` envelope; return exact TX/RX bytes."""
+    results: List[Dict[str, Any]] = []
+    for port in ports:
+        client = LocalDeviceClient(
+            sn=sn, host=host, socket_port=port,
+            key_radix=key_radix, timeout=timeout,
+        )
+        for label, cmd, data in PROBE_VARIANTS:
+            try:
+                out = client.send_probe(cmd, data=data, timeout=timeout)
+            except (ConnectionError, socket.error, TimeoutError) as exc:
+                out = {"cmd": str(cmd), "error": f"{type(exc).__name__}: {exc}",
+                       "tx": b"", "reply_raw": b"", "reply": None,
+                       "envelope": None}
+            out.update({"variant": label, "port": port})
+            results.append(out)
+            if out.get("reply_raw"):
+                break  # device answered — deeper variants can wait
+    return results
+
+
 def main(argv: Optional[List[str]] = None) -> int:
     parser = argparse.ArgumentParser(
         description="Read aGate battery percentage over the device SoftAP socket."
@@ -174,11 +216,13 @@ def main(argv: Optional[List[str]] = None) -> int:
         "--ports", default=",".join(map(str, SOCKET_PORT_CANDIDATES)),
         help="comma-separated socket ports to probe",
     )
-    parser.add_argument("--key-radix", type=int, default=16, choices=(16, 32))
+    parser.add_argument("--key-radix", type=int, default=32, choices=(16, 32))
     parser.add_argument("--timeout", type=float, default=5.0)
     parser.add_argument("--show-hotspot", action="store_true",
                         help="print derived AP SSID/password and exit")
     parser.add_argument("--raw", action="store_true", help="print full reply JSON")
+    parser.add_argument("--probe", action="store_true",
+                        help="send probe variants and print exact TX/RX bytes")
     args = parser.parse_args(argv)
 
     if args.show_hotspot:
@@ -189,6 +233,27 @@ def main(argv: Optional[List[str]] = None) -> int:
         return 0
 
     ports = tuple(int(p) for p in args.ports.split(",") if p.strip())
+    if args.probe:
+        from .protocol import generate_secret_key
+        print(f"secret key for {args.sn}: {generate_secret_key(args.sn)} "
+              f"(radix {args.key_radix})")
+        for out in probe_device(args.sn, host=args.host, ports=ports,
+                                key_radix=args.key_radix, timeout=args.timeout):
+            print(f"--- {out['variant']} -> {args.host}:{out['port']}"
+                  f" cmd={out['cmd']}")
+            if out.get("envelope") is not None:
+                print("  envelope: " + json.dumps(out["envelope"], default=str))
+            tx = out.get("tx") or b""
+            print(f"  tx ({len(tx)} B): {tx.hex()}")
+            rx = out.get("reply_raw") or b""
+            print(f"  rx ({len(rx)} B): {rx.hex() or '(empty)'}")
+            if out.get("reply") is not None:
+                print("  reply: " + json.dumps(out["reply"], default=str))
+                soc = extract_soc(out["reply"])
+                print(f"  soc: {soc['value']} (source={soc['source']})")
+            if out.get("error"):
+                print(f"  error: {out['error']}")
+        return 0
     try:
         result = get_battery(args.sn, host=args.host, ports=ports,
                              key_radix=args.key_radix, timeout=args.timeout)

franklinwh_agate_client/device_local.py

@@ -25,8 +25,10 @@ from .protocol import (
     CommunicationCmd,
     CommunicationObject,
     CommunicationOperate,
+    SequenceNumber,
     SocketFrameDecoder,
-    decrypt_object,
+    decrypt_data,
+    decrypt_request,
     direct_connect_request,
     encrypt_object,
     frame_socket_payload,
@@ -62,8 +64,8 @@ class LocalDeviceClient:
     :param socket_port: local TCP socket port (candidate 18000).
     :param verify: TLS verification for the device API (device likely serves a
         self-signed cert, so ``False`` is the practical default).
-    :param key_radix: key parse radix for the envelope cipher (16 default;
+    :param key_radix: key parse radix for the envelope cipher (32, per the
-        32 is the alternate candidate seen in the disassembly).
+        ``int.tryParse(key, radix: 32)`` call in both directions).
     :param timeout: per-request timeout in seconds.
     """
 
@@ -75,7 +77,7 @@ class LocalDeviceClient:
         scheme: str = "https",
         verify: Optional[bool] = False,
         socket_port: Optional[int] = None,
-        key_radix: int = 16,
+        key_radix: int = 32,
         timeout: float = 10.0,
         extra_headers: Optional[Dict[str, str]] = None,
         session: Optional[requests.Session] = None,
@@ -90,6 +92,7 @@ class LocalDeviceClient:
         self.key_radix = int(key_radix)
         self.timeout = timeout
         self.key = generate_secret_key(sn)
+        self._sn_seq = SequenceNumber()
         self._session = session or requests.Session()
         self._session.headers.update(
             {X_DEVICE_IP: host, **(extra_headers or {})}
@@ -169,19 +172,21 @@ class LocalDeviceClient:
         cmd: CommunicationCmd,
         equip_no: Optional[str] = None,
         operate: CommunicationOperate = CommunicationOperate.QUERY,
-        data: str = "",
+        data: Optional[Dict[str, Any]] = None,
         envelope: Optional[CommunicationObject] = None,
     ) -> Dict[str, Any]:
         """Send one envelope over the local socket and return the parsed reply.
 
-        The request is the app's envelope: compact JSON -> ``encrypt_object``
+        Mirrors ``sendObject``: the envelope's ``snno`` is stamped from the
-        -> int16LE length framing.  The reply is deframed, decrypted (the
+        ``increaseSN`` counter, serialised to compact JSON,
-        unencrypted tail is located via ``cmdType``/``equipNo``) and parsed
+        ``encrypt_object``-ed, and sent **raw** (``_Socket.add``, no length
-        back into a JSON object.
+        prefix).  The reply is accumulated chunk by chunk and run through
+        ``decrypt_data`` until it yields a JSON object.
         """
         if envelope is None:
             envelope = direct_connect_request(
-                cmd, equip_no or self.sn, data=data
+                cmd, equip_no or self.sn, data=data,
+                snno=self._sn_seq.next(),
             )
         return self.send_envelope(envelope)
 
@@ -189,30 +194,82 @@ class LocalDeviceClient:
         wire = encrypt_object(envelope.to_json_string(), self.key,
                               radix=self.key_radix)
         raw = self._socket_roundtrip(frame_socket_payload(wire))
-        reply = decrypt_object(
+        reply = decrypt_data(raw, radix=self.key_radix)
-            raw,
+        if not reply:
-            self.key,
+            raise ConnectionError(
-            cmd_type=envelope.cmd_type,
+                f"undecodable reply ({len(raw)} bytes) from "
-            equip_no=envelope.equip_no,
+                f"{self.socket_host}:{self.socket_port}"
-            radix=self.key_radix,
+            )
-        )
         return json.loads(reply)
 
+    def send_probe(
+        self,
+        cmd: CommunicationCmd,
+        data: Optional[Dict[str, Any]] = None,
+        timeout: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        """Send one envelope, return exact TX/RX bytes plus any parsed reply.
+
+        Never raises on a device-level failure: ``reply`` is the parsed JSON
+        (or ``None``), ``reply_raw`` the exact bytes received (``b""`` when
+        the device accepted the frame but answered with zero bytes).
+        """
+        envelope = direct_connect_request(
+            cmd, self.sn, data=data, snno=self._sn_seq.next(),
+        )
+        wire = encrypt_object(envelope.to_json_string(), self.key,
+                              radix=self.key_radix)
+        reply_raw = self._socket_roundtrip(
+            frame_socket_payload(wire), timeout=timeout, collect_only=True,
+        )
+        reply: Optional[Dict[str, Any]] = None
+        if reply_raw:
+            decoded = decrypt_data(reply_raw, radix=self.key_radix)
+            if decoded:
+                try:
+                    reply = json.loads(decoded)
+                except ValueError:
+                    reply = None
+        return {
+            "cmd": str(cmd),
+            "envelope": envelope.to_dict(),
+            "tx": wire,
+            "reply_raw": reply_raw,
+            "reply": reply,
+        }
+
     def _socket_roundtrip(self, framed: bytes,
-                          timeout: Optional[float] = None) -> bytes:
+                          timeout: Optional[float] = None,
+                          collect_only: bool = False) -> bytes:
         timeout = timeout if timeout is not None else self.timeout
+        received = bytearray()
         with socket.create_connection(
             (self.socket_host, self.socket_port), timeout=timeout
         ) as sock:
+            sock.settimeout(timeout)
             sock.sendall(framed)
             decoder = SocketFrameDecoder()
             while True:
-                chunk = sock.recv(65536)
+                try:
+                    chunk = sock.recv(65536)
+                except socket.timeout:
+                    break
                 if not chunk:
                     break
-                frames = decoder.feed(chunk)
+                received.extend(chunk)
-                if frames:
+                if collect_only:
-                    return frames[0]
+                    # Probe mode: drain what the device sends, then stop on
+                    # the short post-reply idle timeout.  A rejected frame
+                    # is met with zero bytes + close; a real reply arrives
+                    # as buffered bytes first.
+                    sock.settimeout(0.5)
+                    continue
+                if decoder.feed(chunk):
+                    break
+        if collect_only:
+            return bytes(received)
+        if decrypt_data(bytes(received), radix=self.key_radix):
+            return bytes(received)
         raise ConnectionError(
             f"no reply frame from {self.socket_host}:{self.socket_port}"
         )

franklinwh_agate_client/protocol/__init__.py

@@ -15,15 +15,25 @@ from .crc import (
     le_bytes_to_int16,
 )
 from .crypto import (
-    ALT_KEY_RADIX,
     DEFAULT_KEY_RADIX,
+    MARKER,
+    decrypt_data,
     decrypt_object,
+    decrypt_request,
     encrypt_object,
     generate_secret_key,
     header_len_for,
     split_envelope,
 )
-from .envelope import KEY_ORDER, PLACEHOLDER, CommunicationObject, direct_connect_request
+from .envelope import (
+    KEY_ORDER,
+    SNNO_MAX,
+    SNNO_MIN,
+    PLACEHOLDER,
+    CommunicationObject,
+    SequenceNumber,
+    direct_connect_request,
+)
 from .framing import (
     MAGIC_ACK,
     MAGIC_DATA,
@@ -52,15 +62,20 @@ __all__ = [
     "int16_to_le_bytes",
     "le_bytes_to_int16",
     "DEFAULT_KEY_RADIX",
-    "ALT_KEY_RADIX",
+    "MARKER",
     "generate_secret_key",
     "split_envelope",
     "encrypt_object",
     "decrypt_object",
+    "decrypt_request",
+    "decrypt_data",
     "header_len_for",
     "KEY_ORDER",
+    "SNNO_MIN",
+    "SNNO_MAX",
     "PLACEHOLDER",
     "CommunicationObject",
+    "SequenceNumber",
     "direct_connect_request",
     "MAGIC_ACK",
     "MAGIC_DATA",

franklinwh_agate_client/protocol/crypto.py

@@ -1,46 +1,64 @@
-"""Envelope encryption — ``communication_center/core/encryption.dart``.
+"""Envelope cipher — ``communication_center/core/encryption.dart``.
 
 Reconstructed from the blutter disassembly; see
-``docs/RE_DART_FINDINGS.md`` §5 and Open items #2 / #5.
+``docs/RE_DART_FINDINGS.md`` §5.
 
-Wire form of an envelope::
+Request wire form of an envelope::
 
-    s  = compact JSON, e.g.  {"cmdType":0,"equipNo":"SN",...,"data":""}
+    s   = compact JSON, e.g. {"cmdType":25,"equipNo":"SN",...}
-    a  = UTF-8( parts[0] + "," + parts[1] )     # {"cmdType":..,"equipNo":..
+    S1  = parts[0] + "," + parts[1] + ","   # quartet header, trailing comma
-    b  = UTF-8( ",".join(parts[2:]) )           # the rest of the JSON
+    S2  = ",".join(parts[2:])               # the rest of the JSON
-    key_int = int.tryParse(key, radix) or 0
+    a   = UTF-8(S1)                         # sent PLAINTEXT first
-    wire  = bytes( (b[i] + i + len(a) + key_int) & 0xFF ) || a
+    n   = int.tryParse(key, radix: 32) or 0
+    wire = a || bytes((S2[i] + i + len(a) + n) & 0xFF)
 
-i.e. everything after the second comma is transformed byte-wise (position-
+i.e. the two leading JSON parts go on the wire **unchanged first**, and the
-dependent additive stream) and the two leading JSON parts are appended
+tail is transformed byte-wise with a position-dependent additive stream
-**unchanged** at the end.
+(index ``i`` is relative to the tail; ``i + len(a)`` is the absolute JSON
+index). The receiver that knows the serial can compute ``len(a)`` up front
+(``header_len_for``) and split the wire deterministically.
+
+Responses use a different framing (``decrypt_data``): the wire holds the
+plaintext JSON prefix ``s`` (which itself contains the serial's ``X``),
+a literal ``0x58`` separator, then the encoded tail::
+
+    wire = UTF-8(s) || 0x58 || encode(tail)
+
+``decrypt_data`` locates the *second* ``0x58`` byte ``j``, takes
+``s = UTF-8(wire[:j])``, probes ``s + "}"`` as JSON to recover ``equipNo``
+(and hence the key), decodes the tail with the same stream in reverse, and
+returns ``s + "," + tail_plain``.  Anything unexpected yields ``""`` (the
+app's ``decryptData`` returns ``""`` on marker/parse failure and the socket
+layer keeps accumulating bytes).
 
 Confirmed from the disassembly:
+
 * ``generateSecretKey`` = ``sum(utf8(sn)) & 0xFF`` rendered as lowercase hex
   with **no** zero padding (Dart ``_toPow2String(16)``).
-* The key is parsed with ``int.tryParse`` (null -> 0) with a ``radix``
+* The key is parsed with ``int.tryParse(key, radix: 32)`` — the literal
-  argument; a literal ``#0x20`` (32) is materialised right before the call in
+  ``0x20`` (32) is materialised right before the call in both the encrypt
-  both encrypt and decrypt paths.  Hex is self-consistent with the key
+  and decrypt paths.
-  encoding, so **radix 16 is the default** here — pass ``radix=32`` to try the
+* ``encryptObject(object, key)`` takes the object in the first slot and the
-  other candidate.  Confirm against a live capture.
+  key in the second; its result is ``a ++ transform(b)`` (plaintext header
+  first — ``addAll(transformed)`` onto the header copy).
 """
 
 from __future__ import annotations
 
+import json
 from typing import Optional, Tuple
 
-#: Default key radix — see module docstring for the 16-vs-32 note.
+#: Key radix used by both ``encryptObject`` and ``decryptData`` (literal 0x20).
-DEFAULT_KEY_RADIX = 16
+DEFAULT_KEY_RADIX = 32
 
-#: Alternate candidate observed next to the ``tryParse`` call in the asm.
+#: Response separator byte (``0x58``, ASCII ``X``).
-ALT_KEY_RADIX = 32
+MARKER = 0x58
 
 
 def generate_secret_key(equip_no: str) -> str:
     """``generateSecretKey(equipNo)``: sum of UTF-8 bytes, ``& 0xFF``, hex.
 
-    Example: ``generate_secret_key("01234567890123456789")`` is the lowercase
+    Example: ``generate_secret_key("A03X2516013210060005") == "1c"``.
-    hex of ``(sum of those 20 bytes) & 0xFF`` (1- or 2-digit string).
     """
     total = sum(equip_no.encode("utf-8")) & 0xFF
     return format(total, "x")
@@ -55,37 +73,43 @@ def _parse_key(key: str, radix: int) -> int:
 
 
 def split_envelope(json_str: str) -> Tuple[bytes, bytes]:
-    """Split the JSON into ``(a, b)`` as the app does (UTF-8 encoded)."""
+    """Split the JSON into ``(a, b)`` as the app does (UTF-8 encoded).
+
+    ``a`` is the plaintext header ``parts[0] + "," + parts[1] + ","``
+    (trailing comma included); ``b`` is the remaining tail.
+    """
     parts = json_str.split(",")
-    a = (parts[0] + "," + parts[1]).encode("utf-8")
+    a = (parts[0] + "," + parts[1] + ",").encode("utf-8")
     b = ",".join(parts[2:]).encode("utf-8")
     return a, b
 
 
-def encrypt_object(json_str: str, key: str,
+def encrypt_object(
-                   radix: int = DEFAULT_KEY_RADIX) -> bytes:
+    json_str: str, key: str, radix: int = DEFAULT_KEY_RADIX
-    """Encrypt an envelope JSON string into wire bytes (``transform(b) || a``)."""
+) -> bytes:
+    """Encrypt an envelope JSON string into wire bytes (``a || crypt(b)``)."""
     a, b = split_envelope(json_str)
     key_int = _parse_key(key, radix)
     transformed = bytes(
         (byte + i + len(a) + key_int) & 0xFF for i, byte in enumerate(b)
     )
-    return transformed + a
+    return bytes(a) + transformed
 
 
 def header_len_for(cmd_type: int, equip_no: str) -> int:
     """``len(a)`` for an envelope with the given ``cmdType`` / ``equipNo``.
 
     Derived from the *known* key order (``cmdType``, ``equipNo`` first) and
-    compact JSON — this is what a receiver can use to locate the unencrypted
+    compact JSON — this is what a receiver that knows the serial uses to
-    tail of an incoming frame.
+    locate the end of the unencrypted header of an incoming *request* frame.
     """
     part0 = '"cmdType":%d' % int(cmd_type)
     part1 = '"equipNo":"%s"' % equip_no
-    return len(part0) + 1 + len(part1)
+    # parts[0] + "," + parts[1] + "," — plus the leading "{" of the JSON.
+    return len(("{" + part0).encode("utf-8")) + 1 + len(part1.encode("utf-8")) + 1
 
 
-def decrypt_object(
+def decrypt_request(
     wire: bytes,
     key: str,
     header_len: Optional[int] = None,
@@ -95,27 +119,71 @@ def decrypt_object(
 ) -> str:
     """Inverse of :func:`encrypt_object` -> original JSON string.
 
-    ``header_len`` is ``len(a)`` (the unencrypted tail).  When omitted it is
+    Splits the wire at the known plaintext-header length (device side —
-    derived from ``cmd_type`` + ``equip_no`` via :func:`header_len_for` —
+    the device knows its own serial).  ``header_len`` is ``len(a)``; when
-    both must be known in that case.
+    omitted it is derived from ``cmd_type`` + ``equip_no``.
     """
     if header_len is None:
         if cmd_type is None or equip_no is None:
             raise ValueError(
-                "header_len (or cmd_type + equip_no) is required to split the "
+                "header_len (or cmd_type + equip_no) is required to split "
-                "wire buffer"
+                "the wire buffer"
             )
         header_len = header_len_for(cmd_type, equip_no)
-    a = wire[len(wire) - header_len:] if header_len else b""
+    a = wire[:header_len]
-    b = wire[: len(wire) - header_len] if header_len else bytes(wire)
+    b = wire[header_len:]
     key_int = _parse_key(key, radix)
     plain_b = bytes(
         (byte - i - len(a) - key_int) & 0xFF for i, byte in enumerate(b)
     )
-    a_str = a.decode("utf-8")
+    return (a + plain_b).decode("utf-8")
-    b_str = plain_b.decode("utf-8")
+
-    if not a_str:
+
-        return b_str
+# Backwards-compatible alias (device-side request split).
-    if not b_str:
+decrypt_object = decrypt_request
-        return a_str
+
-    return a_str + "," + b_str
+
+def decrypt_data(wire: bytes, radix: int = DEFAULT_KEY_RADIX) -> str:
+    """``decryptData`` — decode one accumulated *response* buffer.
+
+    Faithful port of the app's routine: find the first two ``0x58`` bytes,
+    take ``s`` as everything before the second one, probe ``s + "}"`` for
+    ``equipNo`` to derive the key, decode the tail after the marker, and
+    return ``s + "," + tail``.  Returns ``""`` on any failure (markers
+    missing, probe not JSON, decode error) — the caller keeps accumulating.
+    """
+    try:
+        raw = bytes(wire)
+        i = raw.find(bytes((MARKER,)))
+        if i < 0:
+            return ""
+        j = raw.find(bytes((MARKER,)), i + 1)
+        if j < 0:
+            return ""
+        try:
+            s = raw[:j].decode("utf-8")
+        except UnicodeDecodeError:
+            return ""
+        try:
+            probe = json.loads(s + "}")
+        except ValueError:
+            return ""
+        if not isinstance(probe, dict):
+            return ""
+        equip_no = probe.get("equipNo", "")
+        key = generate_secret_key(str(equip_no))
+        key_int = _parse_key(key, radix)
+        prefix = s + ","
+        prefix_len = len(prefix.encode("utf-8"))
+        tail = raw[j + 1:]
+        plain = bytes(
+            (byte - k - prefix_len - key_int) & 0xFF
+            for k, byte in enumerate(tail)
+        )
+        try:
+            tail_str = plain.decode("utf-8")
+        except UnicodeDecodeError:
+            return ""
+        return prefix + tail_str
+    except Exception:
+        return ""

franklinwh_agate_client/protocol/envelope.py

@@ -1,20 +1,22 @@
 """Wire envelope — ``CommunicationObject``.
 
-JSON key order (exact, from ``toJson`` in ``core/communication_object.dart``):
+Recovered from ``core/communication_object.dart`` (blutter disassembly):
-``cmdType, equipNo, type, timeStamp, snno, len, crc, dataArea, data``.
+
+* ``toJson`` inserts keys in the exact order
+  ``cmdType, equipNo, type, timeStamp, snno, len, crc`` and then appends
+  ``dataArea`` **only when the payload is present and non-empty**.
+  There is **no** ``data`` key on the wire.
+* The constructor takes ``(cmdType, equipNo, {data})``; it zeroes the
+  ``type`` / ``snno`` fields, stamps ``timeStamp`` with the current time in
+  **whole seconds**, and routes ``data`` through the ``data=`` setter, which
+  recomputes ``len`` (UTF-8 byte length of the compact JSON of ``data``)
+  and ``crc`` (``Crc32.calculate`` rendered as **uppercase** hex, no
+  padding). Null/empty data yields ``len == 0``, ``crc == ""`` and no
+  ``dataArea`` key.
+* ``sendObject`` (socket path) overwrites ``snno`` with ``increaseSN()`` —
+  a 1-based counter that wraps at 9999 — before serialising.
 
 See ``docs/RE_DART_FINDINGS.md`` §3.
-
-Confirmed:
-* ``cmdType`` = :class:`CommunicationCmd` raw value
-* ``equipNo`` = device serial number
-* ``type``    = :class:`CommunicationOperate` raw value
-* compact JSON (no whitespace) — matches Dart ``json.encode``
-
-Candidates (not yet confirmed byte-for-byte, see Open items):
-* defaults of ``crc`` / ``dataArea`` — the pool contains the string constant
-  ``"00000000"`` used as a placeholder default;
-* ``len`` semantics (byte length of the payload).
 """
 
 from __future__ import annotations
@@ -24,8 +26,9 @@ import time
 from typing import Any, Dict, Optional
 
 from .commands import CommunicationCmd, CommunicationOperate
+from .crc import crc32
 
-#: Exact key order used by the app's ``toJson``.
+#: Exact key order used by the app's ``toJson`` (``dataArea`` conditional).
 KEY_ORDER = (
     "cmdType",
     "equipNo",
@@ -35,13 +38,50 @@ KEY_ORDER = (
     "len",
     "crc",
     "dataArea",
-    "data",
 )
 
-#: Placeholder default observed in the object pool.
+#: First sequence number issued by ``increaseSN`` (counter starts at 0).
+SNNO_MIN = 1
+#: Counter wraps back to 1 past this value (``> 9999`` check in ``increaseSN``).
+SNNO_MAX = 9999
+
+#: Serial placeholder the ctor substitutes for an empty ``equipNo``.
 PLACEHOLDER = "00000000"
 
 
+def _now_seconds() -> int:
+    """``DateTime`` micros ``~/ 1000 ~/ 1000`` — whole seconds."""
+    return int(time.time())
+
+
+def _compact_json(value: Any) -> str:
+    """Dart ``jsonEncode`` equivalent for our ASCII payloads (compact)."""
+    return json.dumps(value, separators=(",", ":"))
+
+
+def _crc_of_data(data: Dict[str, Any]) -> str:
+    """``calculateCrc``: CRC-32 of the compact JSON, UPPERCASE hex, unpadded."""
+    return format(crc32(_compact_json(data).encode("utf-8")), "X")
+
+
+def _len_of_data(data: Dict[str, Any]) -> int:
+    """``calculateDataLength``: UTF-8 byte length of the compact JSON."""
+    return len(_compact_json(data).encode("utf-8"))
+
+
+class SequenceNumber:
+    """``BaseCommunicationManager`` 1..9999 counter (``increaseSN``)."""
+
+    def __init__(self, start: int = 0) -> None:
+        self._value = int(start)
+
+    def next(self) -> int:
+        self._value += 1
+        if self._value > SNNO_MAX:
+            self._value = SNNO_MIN
+        return self._value
+
+
 class CommunicationObject:
     """One request/response envelope travelling over any transport."""
 
@@ -50,28 +90,38 @@ class CommunicationObject:
         cmd_type: int,
         equip_no: str,
         op_type: int = CommunicationOperate.QUERY,
-        snno: Optional[str] = None,
+        snno: int = 0,
         time_stamp: Optional[int] = None,
-        length: Optional[int] = None,
+        data: Optional[Dict[str, Any]] = None,
-        crc: str = PLACEHOLDER,
-        data_area: str = PLACEHOLDER,
-        data: str = "",
     ) -> None:
         self.cmd_type = int(cmd_type)
-        self.equip_no = str(equip_no)
+        # Empty serial falls back to the "00000000" placeholder (ctor check).
+        self.equip_no = str(equip_no) if equip_no else PLACEHOLDER
         self.op_type = int(op_type)
-        self.snno = snno if snno is not None else str(equip_no)
+        self.snno = int(snno)
-        self.time_stamp = int(time_stamp if time_stamp is not None
+        self.time_stamp = int(
-                              else int(time.time() * 1000))
+            time_stamp if time_stamp is not None else _now_seconds()
-        self.length = int(length) if length is not None else 0
+        )
-        self.crc = str(crc)
+        self.data: Optional[Dict[str, Any]] = None
-        self.data_area = str(data_area)
+        self.length = 0
-        self.data = str(data)
+        self.crc = ""
+        self.set_data(data)
+
+    # ------------------------------------------------------------------ data
+    def set_data(self, data: Optional[Dict[str, Any]]) -> None:
+        """``data=`` setter: stores the payload, recomputes ``len``/``crc``."""
+        self.data = dict(data) if data is not None else None
+        if self.data:
+            self.length = _len_of_data(self.data)
+            self.crc = _crc_of_data(self.data)
+        else:
+            self.length = 0
+            self.crc = ""
 
     # ------------------------------------------------------------------ json
     def to_dict(self) -> Dict[str, Any]:
-        """Fields in the app's exact key order."""
+        """Fields in the app's exact key order (``dataArea`` conditional)."""
-        return {
+        ordered: Dict[str, Any] = {
             "cmdType": self.cmd_type,
             "equipNo": self.equip_no,
             "type": self.op_type,
@@ -79,28 +129,34 @@ class CommunicationObject:
             "snno": self.snno,
             "len": self.length,
             "crc": self.crc,
-            "dataArea": self.data_area,
-            "data": self.data,
         }
+        if self.data:
+            ordered["dataArea"] = self.data
+        return ordered
 
     def to_json_string(self) -> str:
         """Compact JSON string — the input of ``encrypt_object``."""
-        return json.dumps(self.to_dict(), separators=(",", ":"))
+        return _compact_json(self.to_dict())
 
     @classmethod
     def from_dict(cls, raw: Dict[str, Any]) -> "CommunicationObject":
-        """Tolerant parse (missing keys fall back to placeholders)."""
+        """Tolerant parse (missing keys fall back to ctor defaults)."""
-        return cls(
+        obj = cls(
             cmd_type=raw.get("cmdType", 0),
-            equip_no=raw.get("equipNo", ""),
+            equip_no=raw.get("equipNo", "") or "",
             op_type=raw.get("type", 0),
-            snno=raw.get("snno") if raw.get("snno") is not None else None,
+            snno=raw.get("snno", 0) or 0,
             time_stamp=raw.get("timeStamp"),
-            length=raw.get("len"),
+            data=None,
-            crc=raw.get("crc", PLACEHOLDER),
-            data_area=raw.get("dataArea", PLACEHOLDER),
-            data=raw.get("data", ""),
         )
+        data_area = raw.get("dataArea")
+        if isinstance(data_area, dict):
+            obj.set_data(data_area)
+        else:
+            # Preserve observed len/crc when no payload key is present.
+            obj.length = int(raw.get("len", 0) or 0)
+            obj.crc = str(raw.get("crc", ""))
+        return obj
 
     @classmethod
     def from_json_string(cls, s: str) -> "CommunicationObject":
@@ -109,23 +165,29 @@ class CommunicationObject:
     def __repr__(self) -> str:  # pragma: no cover - debugging aid
         return (
             f"CommunicationObject(cmdType={self.cmd_type:#04x}, "
-            f"equipNo={self.equip_no!r}, type={self.op_type})"
+            f"equipNo={self.equip_no!r}, type={self.op_type}, "
+            f"snno={self.snno})"
         )
 
 
 def direct_connect_request(
     cmd: CommunicationCmd,
     equip_no: str,
-    data: str = "",
+    data: Optional[Dict[str, Any]] = None,
+    snno: int = 0,
 ) -> CommunicationObject:
     """Build a *direct-connect* query envelope.
 
-    Mirrors the Dart ``_query*ByDirectConnect`` family
+    Mirrors the Dart ``CommunicationObject.communicate`` factory +
-    (``docs/RE_DART_FINDINGS.md`` — relevant files / RPC layer).
+    ``CommunicationService._query*ByDirectConnect`` family
+    (``docs/RE_DART_FINDINGS.md`` §3).  Note ``snno`` is assigned by
+    ``sendObject`` (see :class:`SequenceNumber`); pass the issued value when
+    re-serialising.
     """
     return CommunicationObject(
         cmd_type=int(cmd),
         equip_no=equip_no,
         op_type=CommunicationOperate.QUERY,
+        snno=snno,
         data=data,
     )

franklinwh_agate_client/protocol/framing.py

@@ -11,6 +11,7 @@
 
 from __future__ import annotations
 
+import json
 from typing import List, Optional, Tuple
 
 from .crc import crc16_hex_string, crc16_modbus, int16_to_le_bytes, le_bytes_to_int16
@@ -130,30 +131,51 @@ def magicEquals(a: bytes, b: bytes) -> bool:
 
 # --------------------------------------------------------------------- socket
 def frame_socket_payload(payload: bytes) -> bytes:
-    """Length-prefixed local-socket frame (candidate — §8).
+    """Local-socket frame: the app sends the encrypted envelope **raw**.
 
-    ``int16LE(len(payload)) || payload`` — the app's socket layer frames with
+    ``ClientSocket.send`` forwards its argument straight to
-    the same little-endian int16 helpers.
+    ``dart:io _Socket.add`` with no length prefix, markers, or handshake
+    (see §8); ``SocketManager._sendData`` only clears the receive buffer
+    afterwards.  This helper is therefore the identity (kept so call sites
+    read like the app).
     """
-    if len(payload) > 0xFFFF:
+    return bytes(payload)
-        raise ValueError("socket payload exceeds the int16 length field")
-    return int16_to_le_bytes(len(payload)) + bytes(payload)
 
 
 class SocketFrameDecoder:
-    """Reassembles length-prefixed socket frames from a byte stream."""
+    """Accumulating socket receiver (``SocketManager._onSocketReceiveData``).
+
+    Each incoming chunk is appended to a growable buffer and ``decrypt_data``
+    is attempted over the whole buffer (inside try/catch in the app); a
+    non-empty decoded string that parses as JSON is one complete reply, at
+    which point the buffer is cleared.  Use :meth:`feed` for raw chunks and
+    :meth:`feed_json` when you want parsed envelopes back.
+    """
 
     def __init__(self) -> None:
         self._buf = bytearray()
 
-    def feed(self, data: bytes) -> List[bytes]:
+    def feed(self, data: bytes) -> List[str]:
-        """Append *data*; return a list of complete payloads (may be empty)."""
+        """Append *data*; return a list of complete decoded JSON strings."""
+        from .crypto import decrypt_data  # local import: cheap, no cycles
+
         self._buf.extend(data)
-        out: List[bytes] = []
+        decoded = decrypt_data(bytes(self._buf))
-        while len(self._buf) >= 2:
+        if not decoded:
-            length = le_bytes_to_int16(bytes(self._buf[:2]))
+            return []
-            if len(self._buf) < 2 + length:
+        try:
-                break
+            json.loads(decoded)
-            out.append(bytes(self._buf[2:2 + length]))
+        except ValueError:
-            del self._buf[2 + length]
+            return []
+        del self._buf[:]
+        return [decoded]
+
+    # Backwards-compatible alias for the old length-prefixed probing API.
+    def feed_frames(self, data: bytes) -> List[bytes]:
+        """Append *data*; return complete **raw wire** buffers (legacy)."""
+        self._buf.extend(data)
+        if not self._buf:
+            return []
+        out = [bytes(self._buf)]
+        del self._buf[:]
         return out

tests/test_protocol.py

@@ -0,0 +1,192 @@
+"""Offline tests for the corrected envelope/cipher/socket wire format.
+
+No radio/hardware needed.  Vectors mirror the blutter disassembly of
+``core/communication_object.dart``, ``core/encryption.dart`` and
+``socket/`` (see ``docs/RE_DART_FINDINGS.md`` §§3, 5, 8).
+"""
+
+import json
+import unittest
+
+from franklinwh_agate_client.protocol import (
+    DEFAULT_KEY_RADIX,
+    CommunicationCmd,
+    CommunicationObject,
+    SequenceNumber,
+    SocketFrameDecoder,
+    decrypt_data,
+    decrypt_request,
+    direct_connect_request,
+    encrypt_object,
+    frame_socket_payload,
+    generate_secret_key,
+    header_len_for,
+    split_envelope,
+)
+from franklinwh_agate_client.protocol.crypto import MARKER
+
+SN = "A03X2516013210060005"
+
+
+def _apower_json(snno=1, time_stamp=1792105196):
+    env = direct_connect_request(
+        CommunicationCmd.DEVICE_APOWER_INFO, SN,
+        data={"update": 2}, snno=snno,
+    )
+    env.time_stamp = time_stamp
+    env.set_data({"update": 2})
+    return env.to_json_string()
+
+
+class TestSecretKey(unittest.TestCase):
+    def test_real_sn_key(self):
+        self.assertEqual(generate_secret_key(SN), "1c")
+
+    def test_key_radix_is_32(self):
+        self.assertEqual(DEFAULT_KEY_RADIX, 32)
+        self.assertEqual(int(generate_secret_key(SN), 32), 44)
+
+    def test_empty_sn_key(self):
+        self.assertEqual(generate_secret_key(""), "0")
+
+
+class TestEnvelope(unittest.TestCase):
+    def test_key_order_no_data_key(self):
+        env = direct_connect_request(
+            CommunicationCmd.DEVICE_APOWER_INFO, SN, data={"update": 2})
+        d = env.to_dict()
+        self.assertEqual(
+            list(d),
+            ["cmdType", "equipNo", "type", "timeStamp", "snno", "len",
+             "crc", "dataArea"],
+        )
+        self.assertNotIn("data", d)
+
+    def test_no_data_means_seven_keys(self):
+        env = direct_connect_request(CommunicationCmd.BMS, SN, data=None)
+        d = env.to_dict()
+        self.assertEqual(
+            list(d),
+            ["cmdType", "equipNo", "type", "timeStamp", "snno", "len", "crc"],
+        )
+        self.assertEqual(env.length, 0)
+        self.assertEqual(env.crc, "")
+
+    def test_empty_map_means_no_data_area(self):
+        env = direct_connect_request(CommunicationCmd.BMS, SN, data={})
+        self.assertNotIn("dataArea", env.to_dict())
+        self.assertEqual(env.length, 0)
+        self.assertEqual(env.crc, "")
+
+    def test_len_crc_auto_uppercase(self):
+        env = direct_connect_request(
+            CommunicationCmd.DEVICE_APOWER_INFO, SN, data={"update": 2})
+        self.assertEqual(env.length, len(b'{"update":2}'))
+        import zlib
+        self.assertEqual(
+            env.crc, format(zlib.crc32(b'{"update":2}'), "X"))
+        self.assertEqual(env.crc, env.crc.upper())
+
+    def test_int_fields(self):
+        env = direct_connect_request(CommunicationCmd.BMS, SN, snno=7)
+        d = env.to_dict()
+        self.assertIsInstance(d["type"], int)
+        self.assertIsInstance(d["timeStamp"], int)
+        self.assertIsInstance(d["snno"], int)
+        self.assertEqual(d["snno"], 7)
+        self.assertEqual(d["type"], 0)  # query
+        # whole seconds, 10 digits
+        self.assertEqual(len(str(d["timeStamp"])), 10)
+
+    def test_snno_sequence_wraps(self):
+        seq = SequenceNumber()
+        self.assertEqual([seq.next(), seq.next()], [1, 2])
+        seq = SequenceNumber(start=9999)
+        self.assertEqual(seq.next(), 1)
+
+    def test_empty_sn_placeholder(self):
+        env = CommunicationObject(0, "")
+        self.assertEqual(env.equip_no, "00000000")
+
+    def test_from_dict_roundtrip(self):
+        env = direct_connect_request(
+            CommunicationCmd.DEVICE_APOWER_INFO, SN, data={"update": 2},
+            snno=3)
+        clone = CommunicationObject.from_json_string(env.to_json_string())
+        self.assertEqual(clone.to_dict(), env.to_dict())
+
+
+class TestCipher(unittest.TestCase):
+    def test_header_plaintext_first(self):
+        wire = encrypt_object(_apower_json(), generate_secret_key(SN))
+        a, b = split_envelope(_apower_json())
+        self.assertEqual(wire[:len(a)], a)
+        self.assertTrue(a.endswith(b","))
+        self.assertEqual(len(wire), len(a) + len(b))
+
+    def test_header_len_matches(self):
+        a, _ = split_envelope(_apower_json())
+        self.assertEqual(header_len_for(0x19, SN), len(a))
+
+    def test_stream_is_absolute_index_plus_n(self):
+        a, b = split_envelope(_apower_json())
+        n = int(generate_secret_key(SN), 32)
+        wire = encrypt_object(_apower_json(), generate_secret_key(SN))
+        tail = wire[len(a):]
+        for i, byte in enumerate(b):
+            self.assertEqual(tail[i], (byte + i + len(a) + n) & 0xFF)
+
+    def test_request_roundtrip(self):
+        body = _apower_json()
+        wire = encrypt_object(body, generate_secret_key(SN))
+        self.assertEqual(
+            decrypt_request(wire, generate_secret_key(SN),
+                            cmd_type=0x19, equip_no=SN),
+            body,
+        )
+
+    def test_decrypt_response_roundtrip(self):
+        body = _apower_json()
+        cut = body.index('"type"')
+        s, tail = body[:cut - 1], body[cut:]
+        n = int(generate_secret_key(SN), 32)
+        prefix_len = len((s + ",").encode("utf-8"))
+        enc = bytes((byte + k + prefix_len + n) & 0xFF
+                    for k, byte in enumerate(tail.encode("utf-8")))
+        resp = s.encode("utf-8") + bytes((MARKER,)) + enc
+        self.assertEqual(decrypt_data(resp), body)
+
+    def test_decrypt_data_failures_are_empty(self):
+        self.assertEqual(decrypt_data(b""), "")
+        self.assertEqual(decrypt_data(b"no markers here"), "")
+        # only one marker
+        self.assertEqual(decrypt_data(b"abc\x58def"), "")
+        # probe not JSON
+        self.assertEqual(decrypt_data(b"\x58garbage\x58\x00\x01"), "")
+
+
+class TestSocketFraming(unittest.TestCase):
+    def test_send_is_raw(self):
+        payload = b"\x01\x02\x03"
+        self.assertEqual(frame_socket_payload(payload), payload)
+
+    def test_decoder_accumulates_until_json(self):
+        body = _apower_json()
+        cut = body.index('"type"')
+        s, tail = body[:cut - 1], body[cut:]
+        n = int(generate_secret_key(SN), 32)
+        prefix_len = len((s + ",").encode("utf-8"))
+        enc = bytes((byte + k + prefix_len + n) & 0xFF
+                    for k, byte in enumerate(tail.encode("utf-8")))
+        resp = s.encode("utf-8") + bytes((MARKER,)) + enc
+        dec = SocketFrameDecoder()
+        # first half: incomplete -> nothing yet
+        self.assertEqual(dec.feed(resp[:len(resp) // 2]), [])
+        # rest: one complete JSON string
+        out = dec.feed(resp[len(resp) // 2:])
+        self.assertEqual(len(out), 1)
+        self.assertEqual(json.loads(out[0])["equipNo"], SN)
+
+
+if __name__ == "__main__":
+    unittest.main()