Is it possible to resize MTD partitions at runtime?

Question

I have a very specific need: to partially replace the content of a flash and to move MTD partition boundaries.

Current map is:

• u-boot 0x000000 0x040000
• u-boot-env 0x040000 0x010000
• kernel 0x050000 0x230000
• initrd 0x280000 0x170000
• scripts 0x3f0000 0x010000
• filesystem 0x400000 0xbf0000
• firmware 0xff0000 0x010000

While desired output is:

• u-boot 0x000000 0x040000
• u-boot-env 0x040000 0x010000
• kernel 0x050000 0x230000
• filesystem 0x280000 0xd70000
• firmware 0xff0000 0x010000

This means to collapse initrd, scripts and filesystem into a single area while leaving the others alone.

Problem is this should be achieved from the running system (booted with the "old" configuration") and I should rewrite kernel and "new" filesystem before rebooting.

The system is an embedded, so I have little space for maneuver (I have a SD card, though).

Of course the rewritten kernel will have "new" configuration written in its DTB.

Problem is transition.

Note: I have seen this Question, but it is very old and it has drawback to need kernel patches, which I would like to avoid.

NOTE2: this question has been flagged for deletion because "not about programming". I beg to disagree: I need to perform said operation on ~14k devices, most of them already sold to customers, so any workable solution should involve, at the very least, scripting.

NOTE3: if absolutely necessary I can even consider (small) kernel modifications (YES, I have means to update kernel remotely).

Answer 1

I will leave the Accepted answer as-is, but, for anyone who happens to come here to find a solution, I want to point out that:

Recent (<4 years old) mtd-utils, coupled with 4.0+ kernel support:

• Definition of a "master" device (MTD device representing the full, unpartitioned Flash). This is a kernel option.
• mtd-utils has a specific mtd-part utility that can add/delete MTD partitions dynamically. NOTE: this utility woks IF (and only if) the above is defined in Kernel.
• With the above utility it's possible to build multiple, possibly overlapping partitions; use with care!

Answer 2

I have three ideas/suggestions:

1. Instead of moving the partitions, can you just split the "new" filesystem image into chunks and write them to the corresponding "old" MTD partitions? This way you don't really need to change MTD partition map. After booting into the new kernel, it will see the new contiguous root filesystem. For JFFS2 filesystem, it should be fairly straightforward to do using split or dd, flash_erase and nandwrite. Something like:

# WARNING: this script assumes that it runs from tmpfs and the old root filesystem is already unmounted.
# Also, it assumes that your shell has arithmetic evaluation, which handles hex (my busybox 1.29 ash does this).

# assuming newrootfs.img is the image of new rootfs
new_rootfs_img="newrootfs.img"

mtd_initrd="/dev/mtd3"
mtd_initrd_size=0x170000
mtd_scripts="/dev/mtd4"
mtd_scripts_size=0x010000
mtd_filesystem="/dev/mtd5"
mtd_filesystem_size=0xbf0000

# prepare chunks of new filesystem image
bs="0x1000"
# note: using arithmetic evaluation $(()) to convert from hex and do the math.
# dd doesn't handle hex numbers ("dd: invalid number '0x1000'") -- $(()) works this around
dd if="${new_rootfs_img}" of="rootfs_initrd"     bs=$(( bs )) count=$(( mtd_initrd_size / bs ))
dd if="${new_rootfs_img}" of="rootfs_scripts"    bs=$(( bs )) count=$(( mtd_scripts_size / bs )) skip=$(( mtd_initrd_size / bs ))
dd if="${new_rootfs_img}" of="rootfs_filesystem" bs=$(( bs )) count=$(( mtd_filesystem_size / bs )) skip=$(( ( mtd_initrd_size + mtd_scripts_size ) / bs ))

# there's no going back after this point

flash_eraseall -j "${mtd_initrd}"
flash_eraseall -j "${mtd_scripts}"
flash_eraseall -j "${mtd_filesystem}"

nandwrite -p "${mtd_initrd}"     rootfs_initrd
nandwrite -p "${mtd_scripts}"    rootfs_scripts
nandwrite -p "${mtd_filesystem}" rootfs_filesystem

# don't forget to update the kernel too

2. There is kernel support for concatenating MTD devices (which is exactly what you're trying to do). I don't see an easy way to use it, but you could create a kernel module, which concatenates the desired partitions for you into a contiguous MTD device.

3. In order to combine the 3 MTD partitions into one to write the new filesystem, you could create a dm-linear mapping over the 3 mtdblocks, and then turn it back into an MTD device using block2mtd. (i.e. mtdblock + device mapper linear + block2mtd) But it looks very awkward and I don't know if it'll work well (for say, OOB data).

EDIT1: added a comment explaining use of $(( bs )) -- to convert from hex as dd doesn't handle hex numbers directly (neither coreutils, nor busybox dd).

Answer 3

AFAIK, @andrey 's answer suggestion 1 is wrong.

an mtd partition is made of a sequence of blocks, any of which could be bad or go bad anytime. this is why the simple mtd char abstraction exists: an mtd char device (not the mtdblock one) is read sequentially and skips bad blocks. nandwrite also writes sequentially and skips bad blocks.

an mtd char device sort of acts like:

• a single file into which you cannot random access, from which you can only read sequentially from the beginning to the end (or to where you get bored).
• a single file into which you cannot random access, to which you can only write sequentially from the beginning (or from an erase block where you previously stopped reading) all the way to the end. (that is, you can truncate and append, but you cannot write mid-file.) to write you need to previously erase all erase blocks from where you start writing to the end of the partition.

this means that the partition size is the maximum theoretical capacity, but typically the capacity will be less due to bad blocks, and can be effectively reduced every time you rewrite the partition. you can never expect to write the full size of an mtd partition.

this is were @andrey 's suggestion 1 is wrong: it breaks up the file to be written into max-sized pieces before writing each piece. but you never know beforehand how much data will fit into an mtd partition without actually writing that data.

instead, you typically need to write some data, and you pray there will be enough good blocks to fit it. if at some point there are not, the write fails and the device reached end-of-life. needless to say, the larger the fraction of a partition you need, the higher the likelihood that the write will fail (and when that happens, it typically means that the device is toast).

to actually implement something akin to suggestion 1, you need to start writing into a partition (skipping bad blocks), and when you run out of erase blocks, you continue writing into the next partition, and so on. the point being: you cannot know where the data boundaries will lay until you actually write the data and fill each partition; there is no other way.