Ad Terras Per Aspera

Small updates to my CD/DVD archival media article

Man, it feels funny updating that after so long. I originally wrote it in 2006, so thats 7 years ago.

USB 3.0 works under Linux

I decided that I needed a real backup solution, even though I have a RAID 5 for file storage in my workstation; maintaining a backup of a 2TB array is a pain in the ass if all you have is blank DVDs.

So, I purchased a Vantec NexStar 3 SuperSpeed (NST-380S3) enclosure, a Samsung EcoGreen F3EG 2TB 5400rpm (HD203WI) drive, and a USB 3 PCI-E controller.

It seems the only shipping USB host controllers the moment all use NEC’s USB 3.0 chip, and almost all the PCI-E boards look alike. They all seem to run in the $25-45 range. The great part is Linux supports NEC’s controller as of 2.6.31. The controller worked with no configuration soon as I put the card in.

I chose that specific Samsung drive because it seems to be the only sane 5400 rpm 2TB drive out there. The only other choices were Seagate’s new 5900 rpm drives (which, according to independent reviews on Newegg and enthusiast forums have an unacceptably high failure rate, very unusual of Seagate), and Western Digital’s Caviar Greens (which are 5400 rpm, but suffer from obsessive head parking which is apparently leading to premature drive failure).

Several reviews peg the HD203WI at an average of 90mb/sec writes for sequential writing, or about 2-3x the speed of USB 2.0.

mkfs.ext4 took 7:44 minutes to create the file system (while iotop confirmed it was doing in excess of 100mb/sec writes for much of the process), and hdparm -t /dev/sdx also indicates the drive in this enclosure can push 100mb/sec.

After writing to the drive for an hour straight, the enclosure is warm but not hot, and after removing the drive from the enclosure, the drive itself is warm; this is compared to the Seagate 7200.12s in my RAID 5 array which could burn you at this point.

Many drives fail in enclosures because they overheat; I don’t think this will happen due to Vantec’s thick aluminum design in the NexStar series enclosures, and the fact that the HW203WI has low power usage.

After formatting with ext4, the file system uses 29GB out of 1.82TB total. Its kind of funny when I’ve owned drives smaller than the space consumed by an empty file system.

I’m rather happy with my purchases overall.

Approximate Youtube Bitrates

I’ve been wondering what bitrates Youtube produces on files, but they don’t upfront say.

New videos are encoded in eight formats. However, due to bug in Youtube, some 24 fps videos (such as those from film sources) will have duplicate frames inserted to make them 30 fps, causing a very noticeable jitter approximately twice a second.

Note: This does not include WebM videos yet as the support is still experimental, and Youtube is not yet encoding videos in 1080p, only 720p (format 45) and 480p (format 43).

Now lets see how a couple high quality videos fair on Youtube.

With these 3 popular HD videos, its easy to tell what sort of bitrate Youtube tries to hit.

Here Men From The Planet Earth First Set Foot Upon the Moon, July 1969 A.D. We Came in Peace For All Mankind

Solid state society: The future of common data storage

Fifty-one years ago, IBM did something amazing, something that changed the world and kick-started the computing revolution twenty years before Intel and Apple and Microsoft and everyone else declared they were open for business: IBM invented the hard drive.

A monster of a machine, a behemoth, one ton of spinning metal the size of a fridge held exactly five megabytes via 50 two foot platters and a bunch of controller hardware and buffer memory. This hard drive was the first of it’s kind, and helped spawn an entire industry of data storage; not only was it faster and easier to use and maintain compared to tape media, it was also expensive and only a few companies could afford this.

The technology over the next few years shrank and increased in performance, and stories of “wash machines” dancing across the data center were well known. More and more companies started buying them to replace or supplement their tape drives, and eventually tape died out in the commercial sector.

Eventually, the three or four home computing revolutions come and go, and the two portable device revolutions come and go. Wash machines become small external units, those external drives become internal (5.25″ full height), and then they become smaller (3.5″) and smaller (2.5″) and smaller (1.8″) yet. Megabytes become gigabytes become tens and hundreds of gigabytes and finally, as of a few months ago, terabytes.

All of this technology ultimately works the same way: spinning platters with magnetic heads reading what an IBM engineer once named “magnetic milkshake.” The one single major flaw in this design is that anything that moves will eventually break down. Spinning drives slower won’t decrease the wear and tear, and neither will cooling them; and new bearing designs? They decrease noise and some wear and tear, but do not prevent mechanical failure.

We’ve invented new technologies, such as redundant arrays of inexpensive disks (RAID) to both increase performance and decrease the chances of mechanical failure eating your data. A suitably sized RAID 6 array can have two drive failures before you risk data loss. An array of, say, six to ten drives for such an array is also huge and outside the realm of most people; and I haven’t seen Apple issue iPods with RAID arrays yet.

In addition to all of this, the magnetic heads have to move across the platter to read and write specific areas, which increase the time it takes to read random data (sequentially read data suffers from this less). If mechanical failure was the major issue of this design, seek times is the secondary issue.

In 1984, a Dr. Fujio Masuoka invented flash memory: a non-volatile memory that can be used as data storage in the same way you’d use tape or hard drives, and flash has no moving parts nor does it use large amounts of power like hard drives do because of spinning platters. You see flash everywhere now, in your cell phones, in your digital cameras, in your hand held game systems, and also in your Wiis. We call drives built out of this technology: solid state drives.

Laptops are now the key target: laptops never have enough power, and battery technology is not keeping pace with our advancements with other technology, and until Santa Rosa more than 3 hour battery life under normal conditions on most laptops was impossible… now it’s simply medium difficulty. Flash technology now has gotten very interesting due to the fact everyone from laptop manufacturers to silent computing aficionados to even the enterprise sector wants flash tech to replace their spinning milkshakes.