i'm working on a SPI NAND Flash applet for #GlasgowInterfaceExplorer.
-
i'm working on a SPI NAND Flash applet for #GlasgowInterfaceExplorer. i would like to add the ability to read (and write) data dumps... however, unlike with SPI NOR flashes where the data is located in an flat address space, SPI NAND devices are structured as pages (each consisting of data + spare + ECC)
how should i format these dumps? is there a standard format? is "use msgpack with a simple custom schema" good enough, considering msgspec promises fast (de)serialization for Python and other languages?
-
@r I would consider this to be a "dogshit experience", correct
-
@r I would consider this to be a "dogshit experience", correct
@whitequark @r encode everything in ASN1 sequences. You get free indications of blocks sizes.
More seriously, why not have a look at floppy formats from hxc? They probably have a format to archive hard drive blocks.
-
i'm working on a SPI NAND Flash applet for #GlasgowInterfaceExplorer. i would like to add the ability to read (and write) data dumps... however, unlike with SPI NOR flashes where the data is located in an flat address space, SPI NAND devices are structured as pages (each consisting of data + spare + ECC)
how should i format these dumps? is there a standard format? is "use msgpack with a simple custom schema" good enough, considering msgspec promises fast (de)serialization for Python and other languages?
@whitequark My preference/bias is against adding dependencies, so JSON-wrapped hexdumps ({"data":..., "spare":..., "ecc":...}, maybe with the address as well) would be fine, with a slight preference for hex over base64 for legibility and ease of decoding, if the size difference isn't an issue. A handy side-effect is that you can (in practice) encode/decode it as a hexdump delmited by fixed strings, so a JSON library isn't strictly necessary (assuming the encoder doesn't change field order/spacing/etc).
Alternatively, you could do something like bin/cue and put the format in another file assuming it's the same for all sectors (though perhaps as JSON instead of a custom text format).
There's also the various disk image formats, some of which might handle sectors of unusual size (I've heard that 512+ecc(?) was a thing), though I don't know if any support annotating the spare/ecc/etc bytes in a sector.
-
@whitequark My preference/bias is against adding dependencies, so JSON-wrapped hexdumps ({"data":..., "spare":..., "ecc":...}, maybe with the address as well) would be fine, with a slight preference for hex over base64 for legibility and ease of decoding, if the size difference isn't an issue. A handy side-effect is that you can (in practice) encode/decode it as a hexdump delmited by fixed strings, so a JSON library isn't strictly necessary (assuming the encoder doesn't change field order/spacing/etc).
Alternatively, you could do something like bin/cue and put the format in another file assuming it's the same for all sectors (though perhaps as JSON instead of a custom text format).
There's also the various disk image formats, some of which might handle sectors of unusual size (I've heard that 512+ecc(?) was a thing), though I don't know if any support annotating the spare/ecc/etc bytes in a sector.
@snowfox do you really want to store 1 GB of data as base64-encoded JSON?
-
@r I would consider this to be a "dogshit experience", correct
@whitequark @r my understanding is that the OS is free to use the extra bytes as they will, so the ostensible 2k+64bytes may actually be used as 4*(512b+16b) or whatever they want. Which means that separating them doesn’t actually give you anything meaningful. Besides which, the OOB data may have important ECC data required to properly interpret the data bytes anyway.
And finally, most tooling expects to receive it in that format (nandwrite, xgecu’s software, etc). -
@r @whitequark *one* of the problems
-
i'm working on a SPI NAND Flash applet for #GlasgowInterfaceExplorer. i would like to add the ability to read (and write) data dumps... however, unlike with SPI NOR flashes where the data is located in an flat address space, SPI NAND devices are structured as pages (each consisting of data + spare + ECC)
how should i format these dumps? is there a standard format? is "use msgpack with a simple custom schema" good enough, considering msgspec promises fast (de)serialization for Python and other languages?
i have settled on "linear image of data+spare bytes, exactly as it came out of the page buffer"
reasoning:
- while data/spare is a well known distinction, it is somewhat arbitrary (spare bytes aren't any different than data bytes with ECC disabled, except for bad block markers)
- moreover, wth ECC enabled, there is more internal structure to the page than just data/spare
- and we can't represent this internal structure because we don't know and the vendor doesn't describe it except free-form in the datasheet
- this is exactly how bytes come out of the device when you ask it
- this is compatible with existing tooling
-
R relay@relay.infosec.exchange shared this topic
