Making USB Bootable Windows 10 Installer

May 3rd, 2018

I needed to update some servers to Windows 10/2016 at work this week, but didn’t have any DVDs large enough to accommodate the 5.8 GB size of the Windows ISO. Normal DVD+Rs are 4.7 GB, so to make use of the ISO I downloaded from MSDN, one needs a DVD-R DL which can hold 8.5 GB of data.

I didn’t feel like going to a store or waiting for delivery, so I used Microsoft’s “Windows 7 USB/DVD Download Tool” (seemingly misnamed since it supports windows 7 and newer).

Once downloaded, the tool is fairly straight forward. You point it to an ISO, and then to a USB drive, and voila, it copies the data.

The annoyance is that I got the error “We were unable to copy your files. Please check your USB device and the selected ISO file and try again.” Trying again, of course, did not resolve the issue.

Fortunately, someone knowledgeable was able to explain the root cause of the error. The USB’s MBR needs to be cleared. This doesn’t happen automatically with the Windows Download Tool.

To clear the MBR and format the drive, follow these steps using the diskpart tool:

diskpart
diskpart> list disk
diskpart> select disk #
diskpart> clean
diskpart> create partition primary
diskpart> select partition 1
diskpart> active
diskpart> format quick fs=fat32
diskpart> assign
diskpart> exit

This resolved my issue and I was able to go along my merry way.

I don’t use Windows a lot in my every-day life, but it is something I use quite a bit at work. Documenting little things like this helps me to remember tricks I come across, and hopefully can help other people searching for solutions to similar problems.

NVidia Web Driver Headless Update

April 6th, 2018

I upgraded my graphics card in my 2008 MacPro to an Nvidia 950. This of course requires the Nvidia WebDriver since Apple doesn’t offer built-in support. I installed the driver, swapped my cards, and it worked like a charm.

Today I had a security update from Apple and thought nothing of it. Unfortunately, after the chime, the video never showed up! I didn’t panic immediately. I SSHed into my computer, and told it to reboot just to see if it was a glitch. No luck… I also tried using screen sharing, alas screen sharing doesn’t work without a GPU.

Luckily it is possible to fix this problem with nothing more than an SSH connection to the troubled Mac. I logged in, and ran the command, system_profiler SPSoftwareDataType, to determine which OS Release I have so I can choose the appropriate WebDriver package.

$ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: OS X 10.11.6 (15G20015)
      Kernel Version: Darwin 15.6.0
      Boot Volume: Macintosh SSD
      Boot Mode: Normal
      Computer Name: MikePro
      User Name: Michael Caldwell (michaelcaldwell)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 3 minutes

The helpful people at MacVidCards.com keep a handy list of all the up-to-date releases of the WebDriver. I selected the appropriate one and was able to install it over SSH using the installer command (which I didn’t know existed before today!).

$ sudo installer -pkg WebDriver-346.03.15f13.pkg -target /
Password:
installer: Package name is NVIDIA Web Driver 346.03.15f13
installer: Upgrading at base path /
installer: The upgrade was successful.
installer: The install requires restarting now.
$ sudo reboot
Connection to 10.0.1.12 closed by remote host.

After the reboot, the display popped on just like it used to. Crisis averted!

More Time Machine Surprises

November 14th, 2017


Another HDD bites the dust… I thought I was prepared. It turns out that there are always unexpected roadblocks when dealing with digital data.

Most recently, I decided to use a RAID 1 scheme (mirroring) as an additional piece of my data security plan. I then have a remote time machine backup. The Time Machine disk is smaller than my RAID, and as such, I only backupĀ  irreplaceable data, like photos and documents to Time Machine. The remainder, like MP3s and DVDs I left to the RAID for protection. In the worst case scenario, I can always re-rip a CD or DVD.

Lesson 1: RAIDs aren’t backups!

Things have been going fine, and I’d been living happily with this setup for about 3 years before disaster strikes. My disk disappeared while I was working, which causes all sorts of weird behaviors. One of the RAID slices failed. The failure was so bad that I couldn’t even read any smart data from the disk. I also couldn’t remount the RAID for some reason. I shut down, removed the failed HDD, and rebooted. The RAID disk reappeared!

But there’s a catch… I did a quick check to see if the data on the disk looks accurate. It turns out that it is not. In fact, it is 6 months out of date! How did that happen?? The only explanation is that RAID failed silently 6 months ago. I’ve been operating on only one disk this whole time!

Lesson 2: Mac OS doesn’t inform of RAID failure!

In fact, disk utility (in El Capitan) doesn’t even show problems with RAID sets at all. The only way to verify a RAID set’s health is to check manually using a command in the terminal, or use the Yosemite Disk Utility.

$ diskutil AppleRAID list

I promptly ordered a new HDD. My plan for recovery is to rsync the data from the out-of-date disk to the new disk, and then rsync the more recent backup from Time Machine on top of it. That will ensure that I have the most coverage of my data.

$ rsync -aPE /Volumes/Old\ Volume/ /Volumes/New\ Volume

When rsyncing, I use the options ‘a’, ‘P’, and ‘E’. The ‘a’ is for ‘archive’ and enables a lot of good switches. ‘P’ is so progress is displayed. This is helpful for gauging how the copy is going. ‘E’ is for extended attributes. I use this because I still have OS 9 files hanging around with resource forks that I am trying to preserve.

The plan goes accordingly, and all my data is merged onto the new HDD! I reboot, login, and then discover the new hiccup; I can’t write anything to my user folder. I can’t load preferences, save anything. I can’t even touch a file!

$ touch ~/test.txt
touch: ~/test.txt: Permission denied

I look at the permissions of my folder to make sure everything looks fine, and indeed it does!

$ ls -ld ~
drwxr-xr-x@ 99 michael staff 3366 13 Sep 14:47 /Users/michael/

I can’t wrap my head around what possible be wrong! If the permissions are right, and the user settings are right? What could the problem be?? My brother is smart enough to have me try listing with the ‘e’ option which shows extended attributes.

$ ls -lde ~
drwxr-xr-x@ 99 michael staff 3366 13 Sep 14:47 /Users/michael/
0: group:everyone deny add_file,delete,add_subdirectory,delete_child,writeattr,writeextattr,chown

Lesson 3: Time Machine puts protective ACLs on everything!

Access Control Lists (ACLs)… I hate ACLs… A wonderful combination of events led to every file being completely protected from all users via ACLs. First of all, I didn’t know this, but Time Machine protects all the backed up files using ACLs. This keeps users from unwittingly deleting things from their backups. It makes sense, but isn’t something I would have expected to be done on a file level. The second ingredient in this predicament is that I used the ‘-E’ option in rsync which copies the extended attributes. If I had left that off, this wouldn’t have been an issue…

Fortunately, there is an easy solution. Remove all the ACLs from everything! This is done with chmod.

$ sudo chmod -RN ~

This takes a while, but works as expected. My user folder is now working as expected, I can write and save documents. I can read my Photos library, etc. But there’s one more issue… I notice that my homebrew is acting funny. I look at my homebrew directory, and discover that none of the symbolic links have had the ACLs removed.

$ ls -le libpng-config 
lrwxr-xr-x+ 1 michael  501  41 Aug 16 22:50 libpng-config@ -> ./Cellar/libpng/1.6.31/bin/libpng-config
 0: group:everyone deny write,delete,append,writeattr,writeextattr,chown

Lesson 4: ACLs can’t be removed from Symbolic Links!

It turns out that using chmod to remove the ACLs from symlinks doesn’t work. The man page suggests that the -h should be used to remove the ACLs.

-h If the file is a symbolic link, change the mode of the link itself rather than the file that the link points to.

But that doesn’t work either.

$ sudo chmod -hN libpng-config 
$ ls -le libpng-config 
lrwxr-xr-x+ 1 michael 501 41 Aug 16 22:50 libpng-config@ -> ../Cellar/libpng/1.6.31/bin/libpng-config
 0: group:everyone deny write,delete,append,writeattr,writeextattr,chown

Searching online yields similar stories. It sounds like this is a long standing bug. Time to pull out the big guns… A one line bash script to erase and recreate all symlinks!

$ find . -type l | while read link; do echo $link; dest=$(readlink "$link"); sudo rm "$link"; ln -s "$dest" "$link"; done

It works like a charm. I run that across all the data that came from Time Machine, and it looks like I am back in business. Full recovery!

I need to change my way of protecting my data. 2 things I am going to do differently going forward:

  1. I won’t rely on RAID 1. The lack of notification if there is a slice failure is unacceptable. I will now use a weekly rsync chron job to sync my data to a second drive in my computer. Having a fully duplicated drive is handy because you can switch to it immediately in the event of a failure, and not have to go through a lengthy restoration process.
  2. Full backup to Time Machine. Although I didn’t suffer (major) data loss, I think it’s worth buying a larger HDD to create a full Time Machine backup, and not just the irreplaceable documents.

So, there you have it. Hopefully this will answer questions for other people out there, especially about ACLs!