Dell Server Workstation iDRAC Dead after Firmware Update Solution R720, R320, R730
This is certainly a poor design, as you can read many seasoned admins who have updated their iDRAC only to have it killed. One possible cause is not by doing all the incremental updates, doing updates from an old iDRAC to one many revisions newer is a sure way to kill things, but even then there is no guarantee based on the amount of failures.
List of threads of people's dead iDRAC's:
https://www.reddit.com/r/homelab/comments/a49b4y/r720_idrac_help/?rdt=40792
https://www.reddit.com/r/homelab/comments/960wx8/dell_r720_idrac_recovery/
https://community.spiceworks.com/t/alert-idrac-not-responding-rebooting/776991
The first solution is don't mass update, as we've seen entire datacenters or racks full of Dell machines with dead iDRAC due to mass update attempts. It is likely not practical to do this fix on a wide scale unless you can migrate the applications to another facility or rack as this requires the servers to be pulled off the rack or at least pulled by the rails and to open the case of course.
Try This First (only if you have lights on the iDRAC):
This solution is for if your iDRAC is totally dead, no flashing lights, dead NIC etc...
You will first need a USB to TTL adapter like this Amazon affil link.
Another successful recovery: http://blog.ignoranthack.me/?p=86
You will see Dell themselves advising to replace the motherboard, but in many cases by connecting to the UART below with a USB to TTL cable, you can tell the iDRAC to reflash from the SDCARD image.
Based off this example of a Dell R720, we can see the UART pins which are labelled by this github user.
This area is at the back right of the server (if looking from the server from the front) and will require you to remove the riser cage to access this area.
Step 1.) Connect to USB to TTL adapter.
Be sure to use the correct pins below (remember the TX side of the TTL goes to the RX side of the UART and the the RX side of the TTL goes to the TX side of the UART). If in doubt, use a multimeter to make sure you aren't connecting to the wrong pins/getting voltage where you shouldn't be.
Step 2.) Recover
Sometimes you may need to go through 2 rounds of bootup procedure by hitting all the annoying keys to get the server booted normally for this to work.
Connect using minicom or screen and run these commands:
Make sure you have the file firmimg.d7 or whatever the name one is from the iDRAC firmware on the SD card that inserts at the front of the server.
util recover -emmc -from_sd -f firmimg.d7 -noreset -clear
Then once the above completes do this:
util reset
Tags:
dell, server, workstation, idrac, firmware, update, seasoned, admins, updated, incremental, updates, revisions, newer, failures, threads, https, www, reddit, homelab, comments, _idrac_help, rdt, wx, dell_r, _idrac_recovery, en, conversations, rack, servers, responding, poweredge, xd, hardware, iidrac, initialization, ccf, ca, rebooting, spiceworks, df, ve, datacenters, racks, attempts, migrate, applications, requires, rails, kbdoc, firmimg, flashing, nic, etc, usb, ttl, adapter, amazon, affil, http, blog, ignoranthack, advising, motherboard, connecting, uart, reflash, sdcard, pins, labelled, github, user, riser, tx, rx, multimeter, aren, voltage, shouldn, bootup, booted, minicom, commands, inserts, util, emmc, from_sd, noreset, completes, reset,