I used the matching 8.3.13 utilities and it didn't work but strangely the newer 8.3.16 which makes DRBD complain works just fine.
GIT-hash: 83ca112086600faacab2f157bc5a9324f7bd7f77 build by root@sighted, 2012-10-09 12:47:51
0: cs:SyncSource ro:Secondary/Primary ds:UpToDate/Inconsistent A r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:5236960
&am........
heartbeat is stopped for some reason
Anyway hnode2 was active and the services are running fine but I see heartbeat has been stopped somehow.
Here is the last log I see of heartbeat:
[quote:23c84415f5]
Sep 9 17:15:32 hnode2 heartbeat: [16738]: info: MSG stats: 9/1762471 ms age 0 [pid16738/MST_CONTROL]
Sep 9 17:15:32 hnode2 heartbeat: [16738]: info: cl_malloc stats: 716/51784021 152624/74519 [pid16738/MST_CONTROL]
Sep 9 17:15:32........