Since my last blog post, a few interesting things happened. The owner of a reverse logistics company who goes by @cookie reached out to me with an interesting problem.
He owned close to 10,000 of these Fastmiles, and needed to update them to later firmware versions prior to sale as earlier firmware versions are atrocious for stability.
Initially, I tried proxying traffic bound for Optus' TR-069 server through through my own Fastmile situated in Optus' network. This worked, but it wasn't quick enough to deal with the scale.
Eventually, I gave up on this approach and decided to try use the hard coded credential identified from the last blog post and perform firmware updates via the UI with a bit of browser automation.
I had concerns that the hardcoded password wouldn't be the same across all devices but by partaking in @cookie's project we learnt several things:
- Out of the 10,000 ish units encountered, every single one had the hardcoded password we found from the last blog post, absolutely insane.
- These fastmiles seem to be sensitive to firmware upgrade paths - more on this later.
Nokia WHY... How NOT to Patch...
When I informed Nokia that all Fastmiles with a firmware version older than 3TG00118ABGA17 came with the same secret admin account password, I expected the solution to be obvious. Disable the account, or generate the password based off things like the Serial number of the device.
Around August/October 2022, a new firmware version 3TG00118ABGA31 got rolled out to my device. I could no longer log into the web interface as admin using the old password, awesome, they fixed it I thought. Oh how wrong I was.
Recently I had a chance to take a look at what they'd done. I started by doing the laziest thing possible, as I had 2 devices, I patched both of them, and just thought I'd retrieve the config for both of the devices to compare them.
THEY BOTH HAVE THE SAME PASSWORD.
I haven't done a full diff to see what's different but on the surface:
- They changed the hardcoded password with another hardcoded password that's longer...
- When logged in as admin, the config/backup restore function is gone.
The admin password for firmware version 3TG00118ABGA31 has been changed to: ZCAFwMyuj#g8-D3K2@v5
EDIT 05/2023: This password seems to be generated from the password you set prior to upgrading to 3TG00118ABGA31, in my case both my units were set with the same password. If someone can provide extract their config following an upgrade without having changed the last one that would be appreciated.
EDIT 11/2023: Cheers to @needmaps for this one. The new password if you didn't update your password previously should be: F1P3@s+6!H
When I told them that changing the password doesn't solve this issue, they decided to fixate on how I retrieved the password again.
Then after a bit longer it seems that they gave up on fixing it.
Unless they change their mind, there's a good chance this new hardcoded password is here to stay. If you own one of these devices, I suggest you login yourself with the credentials above and change it yourself to something unique.
Anywho, moving on from that.
Through working with @cookie we now also know a bit more about the firmware versions available for the Fastmile. I've compiled a table of the versions we've observed thus far and where possible have provided a download link for your consumption.
|Firmware File Name
|Admin password ZCAFwMyuj#g8-D3K2@v5
|ADB disabled, Admin password Nq+L5st7o
|Can upgrade to ABGA17
|Can upgrade to ABAD52
|Cannot upgrade to ABAD52
|Cannot upgrade to ABAD52
If you do happen to end up with one of these devices it should be noted that they seem to be pretty picky with firmware update paths.
- 2019 era firmware versions come with a white UI whereas the later versions come with a dark UI.
- Most seem to need updating to 3TG00118ABAD52 before then being updated to any later versions.
Separately, if you have a Fastmile or other embedded device that you want to extract firmware from, I've decided to write more about this process.
While there's snippets of information spread across various forums about how to do this, none of this was super obvious to me. I swear most forum threads ended something like this instead of actually being helpful:
Broadcom Firmware Dumping via CFE Bootloader
In the first part of this series, I wrote about how you could connect to the serial port of the router. This enables us to dump firmware/config off the router side of the Fastmile. While there are tools to help with this, they seemed to be in varying states of unreliable so I opted to do everything manually.
When connected via serial, the device conveniently prints out the partition table which will be useful for us.
For demonstration purposes we'll be dumping the cfgfs partition which contains things like the configuration files on the Fastmile.
If we want cfgfs, this means we want to extract all data between addresses: 0x000008800000-0x000009000000
If you reboot the device again while spamming any key you can interrupt the boot process to get into the CFE bootloader (this only applies to devices which have unlocked CFE bootloaders).
This exposes several commands. For the purposes of dumping partitions off the device
dn is the one we're interested in.
Figuring out Block Numbers/Pages
NAND flash is laid out in Dies -> Planes -> Blocks -> Pages, ref. Without diving too deep into the topic, what you practically need to know is that:
dncommand expects you to tell it what block/memory address to start dumping from along with how many subsequent pages to look at.
- The Fastmile has 2048 blocks with 64 pages a block.
- I figured out the above through trial and error:
- Attemping to dump a large block number results in an error "Attempt to read block number (5000) beyond the nand max blk(2047), hence there are 2048 blocks.
- Attemping to dump block 0 page 64 (index of 0) using
dnwill actually dump block 1 page 0, meaning each block has 64 pages.
Now we need to know how many pages to view starting from 0x8800000.
- By running
dn 0x8800000 1(dump 1 page from memory address 0x8800000) the tool tells us that 0x8800000 is block 1088.
- By running
dn 0x9000000 1(dump 1 page from memory address 0x9000000) the tool tells us that 0x9000000 is block 1152.
- Subtracting block 1152-1088 means we want to dump 64 blocks.
- With 64 pages per block we are looking at dumping a total of 4096 pages.
Therefore to dump the cfgfs partition the final command we want is:
dn 1088 0 4096
Logging and Converting Output
dn command prints output in a hexdump like format. We want to capture this, remove things from the log we don't need and then convert it to a binary file for final file extraction.
Capturing the output is pretty simple. I used Putty to do this for me.
With logging setup, run the
dn command from above to dump the pages and wait.
Once the capture is complete we need to filter the log for information we don't care about. The goal is to remove lines from the log that we don't care about e.g.:
- White space lines.
- Other descriptive lines (e.g. ------ block blah).
- Spare area data for each page.
- Any NAND read errors.
- In my case the entirety of block 1117 needed to be excluded.
I wrote a small script to do this for me (you'll want to adjust my exclusion of block 1117 to a more generic error message if you're going to reuse it).
The cleaned output can be obtained like so:
python3 parse.py dump.log > ~/Workspace/Fastmile/Config/output.log
We need to remove the first column and last column from our output log keeping only columns 2-5 and then convert it to a binary file. This blog here explains in detail what we're trying to do (https://nstarke.github.io/hexdump/binary/linux/2021/08/27/hexdump-to-binary.html).
The modified oneliner for us looks like this:
cut -d' ' -f2-5 output.log | xxd -r -p > output.bin
Finally, with a bit of luck, you can simply run ubidump on the binary file to extract files.
Taking a browse through we see files like the config_encryption.cfg which is the configuration file that used to be accessible through the Web UI.
Android Firmware Dumping via EDL
Shifting our attention to the Android side of the device. EDL stands for Emergency DownLoad Mode and is a special boot mode normally used by OEMs to allow recovery of bricked devices, force flashes of firmware or dumping of firmware.
The Fastmile can be put into this mode to obtain a live copy of the Android side's firmware in case the device becomes bricked or you want to analyse it.
To place the Fastmile into EDL mode, two headers (next to the UART ones) need to be shorted on boot. The power switch button pictured was just used to more conveniently short those 2 pins, but you can use a screwdriver to do this as well.
When plugging in a USB-C cable you should then see a device with ID 05c6:9008 appear when you run lsusb.
From here you can use tools like: https://github.com/bkerler/edl to dump LUNs/partitions from the device and write them back.
edl printgpt --memory=ufs --lun=0 lists partitions for LUN0 on the fastmile.
edl rf lun0.bin --memory=ufs --lun=0 can be used to dump the first LUN to file.
Other commands: https://github.com/bkerler/edl#for-ufs-flash